Sep 14 04:44:50.647141 ip-10-0-137-105 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 04:44:50.647159 ip-10-0-137-105 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 04:44:50.647170 ip-10-0-137-105 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 04:44:50.647646 ip-10-0-137-105 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 04:45:02.020960 ip-10-0-137-105 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 04:45:02.020980 ip-10-0-137-105 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0184246ba8174f3f95e8a1c34245288e --
Sep 14 04:46:45.537243 ip-10-0-137-105 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 04:46:45.956696 ip-10-0-137-105 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 04:46:45.956696 ip-10-0-137-105 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 04:46:45.956696 ip-10-0-137-105 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 04:46:45.956696 ip-10-0-137-105 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 04:46:45.956696 ip-10-0-137-105 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 04:46:45.960367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.960267    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966372    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966396    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966400    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966403    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966406    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966409    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.966400 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966412    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966415    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966418    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966421    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966424    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966427    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966430    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966432    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966437    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966441    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966444    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966447    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966450    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966455    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966460    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966464    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966467    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966470    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.966664 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966472    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966475    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966478    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966481    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966484    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966486    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966489    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966492    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966502    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966505    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966508    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966510    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966513    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966515    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966520    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966522    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966525    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966527    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966530    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966533    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.967122 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966535    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966538    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966540    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966543    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966546    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966548    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966551    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966553    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966556    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966559    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966561    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966564    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966566    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966569    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966572    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966575    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966577    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966580    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966583    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966586    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.967600 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966589    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966591    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966594    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966596    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966599    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966601    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966604    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966608    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966611    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966613    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966616    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966618    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966621    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966623    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966626    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966629    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966631    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966634    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966637    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966640    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.968104 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966642    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966645    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966647    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966650    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966652    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966654    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966657    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966660    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966664    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966666    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966669    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966672    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966674    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966677    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.966680    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967420    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967427    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967430    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967433    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967436    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.968582 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967439    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967441    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967444    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967446    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967449    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967452    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967454    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967457    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967460    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967463    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967466    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967468    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967474    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967478    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967482    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967486    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967489    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967492    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967495    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.969079 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967498    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967500    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967503    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967505    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967508    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967510    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967513    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967515    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967519    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967523    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967526    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967529    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967532    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967535    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967537    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967540    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967543    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967546    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967549    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967552    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.969654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967554    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967557    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967559    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967562    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967564    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967567    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967578    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967581    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967584    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967587    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967589    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967592    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967594    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967597    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967600    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967604    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967606    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967609    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967612    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967614    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.970199 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967618    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967620    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967623    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967626    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967629    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967632    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967635    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967638    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967640    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967643    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967646    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967648    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967651    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967653    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967656    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967658    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967661    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967663    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967666    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.970728 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967669    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967672    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967675    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967678    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967680    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967683    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967685    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967689    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967692    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967695    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967697    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967700    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967702    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967705    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967708    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.967710    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968153    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968159    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968162    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968165    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.971209 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968168    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968171    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968174    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968177    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968180    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968182    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968185    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968188    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968190    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968193    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968195    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968198    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968201    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968203    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968206    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968208    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968211    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968213    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968216    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968218    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.971689 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968221    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968224    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968228    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968232    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968235    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968238    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968241    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968244    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968247    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968250    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968253    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968256    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968259    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968262    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968265    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968267    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968270    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968273    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968275    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.972221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968278    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968282    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968286    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968291    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968294    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968297    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968300    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968303    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968306    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968308    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968311    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968314    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968316    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968319    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968322    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968324    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968327    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968329    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968332    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.972693 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968335    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968339    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968342    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968345    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968347    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968350    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968352    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968355    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968358    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968360    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968363    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968365    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968368    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968370    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968372    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968375    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968378    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968380    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968383    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968386    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.973184 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968389    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968391    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968393    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968396    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968398    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968401    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968403    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968406    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968408    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968411    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968413    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968416    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968418    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968425    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968427    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968430    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.968432    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968516    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968559    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968567    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968572    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 04:46:45.973712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968576    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968580    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968584    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968589    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968592    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968595    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968599    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968603    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968606    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968609    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968612    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968614    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968618    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968621    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968625    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968628    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968631    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968634    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968637    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968641    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968644    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968647    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968650    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968653    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 04:46:45.974265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968658    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968661    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968664    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968667    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968671    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968674    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968677    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968681    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968684    2571 flags.go:64] FLAG: --enable-server="true"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968687    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968692    2571 flags.go:64] FLAG: --event-burst="100"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968695    2571 flags.go:64] FLAG: --event-qps="50"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968698    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968701    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968704    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968708    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968711    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968714    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968717    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968720    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968723    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968727    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968730    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968733    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968735    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 04:46:45.974896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968738    2571 flags.go:64] FLAG: --feature-gates=""
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968742    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968746    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968749    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968752    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968755    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968758    2571 flags.go:64] FLAG: --help="false"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968762    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968766    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968769    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968771    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968775    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968778    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968782    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968785    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968788    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968791    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968794    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968797    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968800    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968803    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968805    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968808    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968811    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 04:46:45.975585 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968814    2571 flags.go:64] FLAG: --lock-file=""
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968817    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968819    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968822    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968828    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968831    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968834    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968837    2571 flags.go:64] FLAG: --logging-format="text"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968840    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968843    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968846    2571 flags.go:64] FLAG: --manifest-url=""
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968848    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968853    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968856    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968860    2571 flags.go:64] FLAG: --max-pods="110"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968864    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968867    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968870    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968873    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968876    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968879    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968882    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968891    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968894    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968897    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 04:46:45.976197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968900    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968903    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968909    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968912    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968915    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968918    2571 flags.go:64] FLAG: --port="10250"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968921    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968924    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01b8087cc233bf853"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968945    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968949    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968952    2571 flags.go:64] FLAG: --register-node="true"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968956    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968961    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968964    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968967    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968970    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968974    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968977    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968980    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968983    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968986    2571 flags.go:64] FLAG: --runonce="false"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968989    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968992    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.968997    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969000    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 04:46:45.976834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969003    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969007    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969010    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969013    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969018    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969021    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969024    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969027    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969030    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969033    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969036    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969041    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969044    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969047    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969051    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969054    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969057    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969060    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969063    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969066    2571 flags.go:64] FLAG: --v="2"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969071    2571 flags.go:64] FLAG: --version="false"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969076    2571 flags.go:64] FLAG: --vmodule=""
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969080    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.969084    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969208    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.977493 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969212    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969215    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969218    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969220    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969223    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969227    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969230    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969232    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969235    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969238    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969240    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969244    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969247    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969252    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969255    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969258    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969261    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969263    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969266    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969269    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.978123 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969271    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969274    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969276    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969279    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969281    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969284    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969286    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969289    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969293    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969297    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969301    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969304    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969307    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969310    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969313    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969316    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969318    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969322    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969325    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.978610 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969329    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969332    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969335    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969338    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969340    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969343    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969346    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969348    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969351    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969354    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969356    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969359    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969362    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969364    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969367    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969369    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969372    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969374    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969377    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969379    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.979162 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969382    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969385    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969387    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969390    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969393    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969395    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969398    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969400    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969403    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969407    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969412    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969415    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969417    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969419    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969422    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969425    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969427    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969430    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969432    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969435    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.979654 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969437    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969440    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969442    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969445    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969448    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969450    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969453    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969455    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969458    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969461    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969463    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969466    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969468    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969471    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969473    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969476    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969479    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969481    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.980173 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.969484    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.970169    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.978308    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.978332    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.978352    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.978358    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978424    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978429    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978432    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978435    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978438    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978441    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978443    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978447    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978450    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.980659 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978452    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978455    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978458    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978460    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978463    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978466    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978468    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978471    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978473    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978476    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978478    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978481    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978484    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978486    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978489    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978491    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978493    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978496    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978498    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978501    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.981071 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978510    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978513    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978516    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978520    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978524    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978527    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978531    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978533    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978536    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978539    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978541    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978544    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978547    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978549    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978552    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978554    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978557    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978560    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978563    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.981562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978566    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978569    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978571    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978573    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978577    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978581    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978584    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978587    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978590    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978592    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978595    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978597    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978600    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978603    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978611    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978614    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978617    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978620    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978623    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.982076 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978625    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978628    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978630    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978633    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978635    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978638    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978640    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978643    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978646    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978648    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978650    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978653    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978655    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978658    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978661    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978663    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978666    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978668    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978671    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978673    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.982596 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978677    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978681    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978684    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978686    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978689    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978692    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978695    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978698    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978706    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978709    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978711    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978714    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.978720    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978865    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978871    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 04:46:45.983150 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978874    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978876    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978879    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978882    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978884    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978887    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978889    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978892    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978895    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978898    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978900    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978903    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978905    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978909    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978914    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978917    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978920    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978922    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978925    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 04:46:45.983549 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978945    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978950    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978953    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978956    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978958    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978961    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978964    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978973    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978976    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978979    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978983    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978986    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978989    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978992    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978995    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.978998    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979001    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979003    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979006    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 04:46:45.984037 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979008    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979011    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979013    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979016    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979019    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979021    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979023    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979026    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979028    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979031    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979034    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979036    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979039    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979041    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979044    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979046    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979049    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979051    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979054    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979056    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 04:46:45.984492 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979059    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979069    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979071    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979074    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979077    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979079    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979082    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979085    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979087    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979090    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979092    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979095    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979097    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979100    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979102    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979105    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979107    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979110    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979112    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979115    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 04:46:45.984993 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979117    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979120    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979123    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979126    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979128    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979130    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979134    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979137    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979141    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979143    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979146    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979148    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979151    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979153    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979162    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979164    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979167    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979169    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 04:46:45.985496 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:45.979172    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.979177    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.979332    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.981952    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.983062    2571 server.go:1019] "Starting client certificate rotation"
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.983193    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.983229    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 04:46:45.986050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.985898    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 04:46:45.988211 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.988186    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 04:46:45.996277 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.996250    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 14 04:46:45.999966 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:45.999917    2571 log.go:25] "Validated CRI v1 image API"
Sep 14 04:46:46.000943 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.000900    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 04:46:46.007195 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.007170    2571 fs.go:135] Filesystem UUIDs: map[6558de25-a999-49e9-9aac-ea8859ab7e31:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f15652cd-4a1a-42c8-853e-116d501a890d:/dev/nvme0n1p3]
Sep 14 04:46:46.007318 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.007192    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 04:46:46.013724 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.013589    2571 manager.go:216] Machine: {Timestamp:2026-09-14 04:46:46.012129507 +0000 UTC m=+0.374081941 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099896 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d5f8a4d4e6d1ff31960cef66d97bd SystemUUID:ec2d5f8a-4d4e-6d1f-f319-60cef66d97bd BootID:0184246b-a817-4f3f-95e8-a1c34245288e Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:47:18:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:47:18:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:7d:4f:29:ec:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 04:46:46.013724 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.013716    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 04:46:46.013890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.013876    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 04:46:46.017970 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.017911    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 04:46:46.018139 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.017975    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 04:46:46.019462 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.019442    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 04:46:46.019652 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.019640    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 04:46:46.019692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.019656    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 04:46:46.019692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.019665    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 04:46:46.020376 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.020365    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 04:46:46.021720 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.021707    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 04:46:46.026506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026491    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 04:46:46.026646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026636    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 04:46:46.026679 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026671    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 04:46:46.026709 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026686    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 04:46:46.026740 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026710    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 04:46:46.026782 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026770    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 14 04:46:46.026824 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.026806    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 04:46:46.027275 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.027261    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 04:46:46.027275 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.027281    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 04:46:46.029962 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.029943    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 04:46:46.032966 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.032949    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 04:46:46.033008 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.032983    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 04:46:46.034477 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.034445    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 04:46:46.034585 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.034520    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 04:46:46.035220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035205    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035230    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035236    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035242    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035251    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035256    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 04:46:46.035268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035270    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 04:46:46.035427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035276    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 04:46:46.035427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035289    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 04:46:46.035427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035295    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 04:46:46.035427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035303    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 04:46:46.035427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.035309    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 04:46:46.036196 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.036186    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 04:46:46.036196 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.036197    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 04:46:46.040492 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.040473    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 04:46:46.041253 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.041238    2571 server.go:1268] "Started kubelet"
Sep 14 04:46:46.041332 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.041301    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 04:46:46.041387 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.041340    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 04:46:46.041439 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.041408    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 04:46:46.042198 ip-10-0-137-105 systemd[1]: Started Kubernetes Kubelet.
Sep 14 04:46:46.042964 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.042810    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 04:46:46.043822 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.043805    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 04:46:46.045648 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.045627    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 04:46:46.047117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.047102    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ztj5"
Sep 14 04:46:46.047496 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.046290    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-105.ec2.internal.18d5168954f2e245  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-105.ec2.internal,UID:ip-10-0-137-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-105.ec2.internal,},FirstTimestamp:2026-09-14 04:46:46.041174597 +0000 UTC m=+0.403127030,LastTimestamp:2026-09-14 04:46:46.041174597 +0000 UTC m=+0.403127030,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-105.ec2.internal,}"
Sep 14 04:46:46.052356 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.052336    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 04:46:46.053079 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053062    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 04:46:46.053183 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.053143    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.053800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053758    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 04:46:46.053800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053780    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 04:46:46.053997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053758    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 04:46:46.053997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053867    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 04:46:46.053997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.053873    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 04:46:46.054554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054539    2571 factory.go:153] Registering CRI-O factory
Sep 14 04:46:46.054554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054557    2571 factory.go:223] Registration of the crio container factory successfully
Sep 14 04:46:46.054669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054614    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 04:46:46.054669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054622    2571 factory.go:55] Registering systemd factory
Sep 14 04:46:46.054669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054628    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 14 04:46:46.054669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054644    2571 factory.go:103] Registering Raw factory
Sep 14 04:46:46.054669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.054661    2571 manager.go:1196] Started watching for new ooms in manager
Sep 14 04:46:46.055630 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.055601    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6ztj5"
Sep 14 04:46:46.056225 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.056174    2571 manager.go:349] Starting recovery of all containers
Sep 14 04:46:46.065556 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.065528    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:46.068134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.068114    2571 manager.go:354] Recovery completed
Sep 14 04:46:46.070313 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.070285    2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-105.ec2.internal\" not found" node="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.073468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.073452    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.075606 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.075581    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.075702 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.075622    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.075702 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.075633    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.076291 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.076278    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 04:46:46.076291 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.076285    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.076291 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.076291    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 04:46:46.076460 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.076313    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 04:46:46.077091 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.077077    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.077157 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.077106    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.077157 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.077117    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.078579 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.078564    2571 policy_none.go:49] "None policy: Start"
Sep 14 04:46:46.078655 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.078584    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 04:46:46.078655 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.078599    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 04:46:46.080706 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.080691    2571 policy_none.go:47] "Start"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.128830    2571 manager.go:337] "Starting Device Plugin manager"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.129073    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129088    2571 server.go:87] "Starting device plugin registration server"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129413    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129426    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129532    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129723    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.129733    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.130303    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 04:46:46.134509 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.130355    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.186001 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.185963    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 04:46:46.187315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.187287    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 04:46:46.187366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.187359    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 04:46:46.187398 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.187387    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 04:46:46.187502 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.187486    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 04:46:46.190109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.190079    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:46.230119 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.230007    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.231187 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.231168    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.231315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.231199    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.231315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.231212    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.231315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.231244    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.240594 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.240573    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.240651 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.240599    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-105.ec2.internal\": node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.258108 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.258081    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.288477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.288437    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"]
Sep 14 04:46:46.288627 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.288527    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.289612 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.289586    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.289719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.289626    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.289719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.289642    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.291997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.291985    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.292145 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.292185 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292156    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.292861 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292846    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.292861 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292855    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.292861 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292867    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.293049 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292875    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.293049 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292879    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.293049 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.292889    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.295254 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.295240    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.295332 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.295268    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 04:46:46.296146 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.296132    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 04:46:46.296212 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.296160    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 04:46:46.296212 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.296170    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeHasSufficientPID"
Sep 14 04:46:46.327855 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.327819    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-105.ec2.internal\" not found" node="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.332488 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.332469    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-105.ec2.internal\" not found" node="ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.358390 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.358358    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.454459 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.454426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.454459 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.454456    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.454626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.454474    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4cd3bb05a5d106b39c55a9ac721c0cb9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-105.ec2.internal\" (UID: \"4cd3bb05a5d106b39c55a9ac721c0cb9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.458488 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.458465    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.554810 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554724    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4cd3bb05a5d106b39c55a9ac721c0cb9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-105.ec2.internal\" (UID: \"4cd3bb05a5d106b39c55a9ac721c0cb9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.554810 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554745    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4cd3bb05a5d106b39c55a9ac721c0cb9-config\") pod \"kube-apiserver-proxy-ip-10-0-137-105.ec2.internal\" (UID: \"4cd3bb05a5d106b39c55a9ac721c0cb9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.554810 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.555018 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554834    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.555018 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554872    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.555018 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.554879    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb86b5cc2be15089cefdf4c423984a58-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal\" (UID: \"cb86b5cc2be15089cefdf4c423984a58\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.558763 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.558746    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.631000 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.630958    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.634854 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.634832    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:46.659008 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.658976    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.759538 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.759496    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.860155 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.860076    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.960835 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:46.960804    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:46.983011 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.982979    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 04:46:46.983159 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.983124    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 04:46:46.983201 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:46.983159    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 04:46:47.061180 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:47.061147    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:47.061321 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.061170    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 04:41:45 +0000 UTC" deadline="2028-02-19 02:28:55.824044369 +0000 UTC"
Sep 14 04:46:47.061321 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.061210    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12549h42m8.762837824s"
Sep 14 04:46:47.162013 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:47.161978    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-105.ec2.internal\" not found"
Sep 14 04:46:47.236254 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.236223    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:47.253835 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.253811    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:47.262139 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:47.262098    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb86b5cc2be15089cefdf4c423984a58.slice/crio-76fadb55aba6ae57bbb3a772b13a03c86cde187336d76ab787937d4f0f4ca3ae WatchSource:0}: Error finding container 76fadb55aba6ae57bbb3a772b13a03c86cde187336d76ab787937d4f0f4ca3ae: Status 404 returned error can't find the container with id 76fadb55aba6ae57bbb3a772b13a03c86cde187336d76ab787937d4f0f4ca3ae
Sep 14 04:46:47.262385 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:47.262352    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd3bb05a5d106b39c55a9ac721c0cb9.slice/crio-964e02ee17a81e3b228683433ba071176cd169622f9033c3278f65383f236696 WatchSource:0}: Error finding container 964e02ee17a81e3b228683433ba071176cd169622f9033c3278f65383f236696: Status 404 returned error can't find the container with id 964e02ee17a81e3b228683433ba071176cd169622f9033c3278f65383f236696
Sep 14 04:46:47.265137 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.265118    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 04:46:47.265983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.265966    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 04:46:47.266450 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.266434    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"
Sep 14 04:46:47.274628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.274608    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 04:46:47.361471 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:47.361392    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:48.027751 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.027718    2571 apiserver.go:52] "Watching apiserver"
Sep 14 04:46:48.035397 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.035370    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 04:46:48.035765 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.035736    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-7zh6h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal","openshift-multus/multus-additional-cni-plugins-4m8tl","openshift-network-diagnostics/network-check-target-fxr6b","kube-system/konnectivity-agent-jpwnd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67","openshift-cluster-node-tuning-operator/tuned-2hjkd","openshift-dns/node-resolver-hnm2q","openshift-multus/multus-qm95s","openshift-multus/network-metrics-daemon-9nbn5","openshift-network-operator/iptables-alerter-kvbhv","openshift-ovn-kubernetes/ovnkube-node-n7vbm","kube-system/global-pull-secret-syncer-n7j5q","kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal"]
Sep 14 04:46:48.038282 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.038257    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.040221 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.040200    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.040338 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.040275    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:48.041074 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.040895    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.041220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.041195    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qdzj5\""
Sep 14 04:46:48.041322 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.041256    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.042360 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.042245    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.044322 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044289    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:48.044407 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.044351    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:48.044463 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044438    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.044463 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044454    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 04:46:48.044589 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044576    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gltrn\""
Sep 14 04:46:48.044864 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044839    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 04:46:48.045005 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.044989    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.045076 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.045067    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 04:46:48.047617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.047124    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.049524 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.049507    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.049959 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.049796    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 04:46:48.050052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.049974    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2cmpc\""
Sep 14 04:46:48.050114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.050050    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 04:46:48.051564 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.051544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.052019 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.052003    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.052156 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.052142    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 04:46:48.052374 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.052358    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.053220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.053202    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 04:46:48.053316 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.053230    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7g4rv\""
Sep 14 04:46:48.054810 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.053838    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.055911 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.055896    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 04:46:48.056272 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.056255    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.056476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.056459    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2scrm\""
Sep 14 04:46:48.056919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.056903    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 04:46:48.057164 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.057147    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.057671 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.057657    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.057832 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.057815    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xbbzg\""
Sep 14 04:46:48.058194 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.058179    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.058243 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.058228    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:48.060414 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.059676    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.060414 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.059990    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.060414 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.060358    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7rqxz\""
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.061700    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-var-lib-kubelet\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-kubelet-config\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062742    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062766    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-registration-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062859    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb72b\" (UniqueName: \"kubernetes.io/projected/df98b66d-882d-4433-a8a2-170911fd83be-kube-api-access-nb72b\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.062983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.062964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-host\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063002    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cnibin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-kubernetes\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063049    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-run\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cni-binary-copy\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dmsd\" (UniqueName: \"kubernetes.io/projected/804bd645-d44d-436f-afde-bc038ec57fc1-kube-api-access-9dmsd\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.063484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.063761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/804bd645-d44d-436f-afde-bc038ec57fc1-serviceca\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.063790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063758    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-system-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.063820 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:48.063848 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063822    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df98b66d-882d-4433-a8a2-170911fd83be-tmp-dir\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.063899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063852    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-conf\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.063968 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063913    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-binary-copy\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.064024 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.063970    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-bin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.064024 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-daemon-config\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.064109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064040    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-tuned\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.064109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/804bd645-d44d-436f-afde-bc038ec57fc1-host\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.064193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-netns\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.064193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064165    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-device-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.064285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-sys\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.064285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef80e464-bdf3-47ea-b084-bd8f06a17739-agent-certs\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.064285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064252    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef80e464-bdf3-47ea-b084-bd8f06a17739-konnectivity-ca\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.064285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064272    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.064285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-multus-certs\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.064419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064309    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.064419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064333    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.064419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064336    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df98b66d-882d-4433-a8a2-170911fd83be-hosts-file\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.064419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064392    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-lib-modules\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.064544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064422    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5rql\" (UniqueName: \"kubernetes.io/projected/2641c440-5495-414d-8912-829662eccbc4-kube-api-access-d5rql\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.064544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064431    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xch56\""
Sep 14 04:46:48.064544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064485    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc9l7\" (UniqueName: \"kubernetes.io/projected/e94daa96-3583-4bff-870d-cd25f30e49e1-kube-api-access-tc9l7\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.064544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064518    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-modprobe-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.064722 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-cnibin\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.064722 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064657    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.064722 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-socket-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.064887 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064771    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-dbus\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.064964 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064945    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-hostroot\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065001 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.064980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-socket-dir-parent\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065033 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysconfig\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.065061 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhqz9\" (UniqueName: \"kubernetes.io/projected/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-kube-api-access-nhqz9\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.065090 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.065125 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065075    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.065155 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065102    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-systemd\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.065238 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065214    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-system-cni-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.065612 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065593    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-tmp\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.065671 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-k8s-cni-cncf-io\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065707 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065671    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvq9n\" (UniqueName: \"kubernetes.io/projected/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-kube-api-access-jvq9n\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065753 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065704    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.065787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-multus\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065816 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-kubelet\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065835    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-conf-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.065905 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-etc-kubernetes\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.066053 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.065958    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-sys-fs\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.070257 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.069495    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 04:46:48.071858 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.071826    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 04:46:48.072104 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.072070    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 04:46:48.072182 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.072120    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 04:46:48.072399 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.072382    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 04:46:48.073331 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.073128    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 04:46:48.073331 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.073178    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 04:46:48.073482 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.073464    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-894jg\""
Sep 14 04:46:48.074727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.074701    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 04:46:48.085342 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.085319    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jj2js"
Sep 14 04:46:48.100100 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.100077    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jj2js"
Sep 14 04:46:48.154979 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.154953    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 04:46:48.167246 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-daemon-config\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.167246 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167249    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-tuned\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167272    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/804bd645-d44d-436f-afde-bc038ec57fc1-host\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-slash\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-netns\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-device-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-sys\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167393    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef80e464-bdf3-47ea-b084-bd8f06a17739-agent-certs\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167415    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef80e464-bdf3-47ea-b084-bd8f06a17739-konnectivity-ca\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167413    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/804bd645-d44d-436f-afde-bc038ec57fc1-host\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.167468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167439    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-kubelet\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167477    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw8m5\" (UniqueName: \"kubernetes.io/projected/3d32c872-6925-475a-8500-7029bc315861-kube-api-access-cw8m5\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-sys\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4696d571-6afb-4978-9597-86e1e0a2fe64-iptables-alerter-script\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-multus-certs\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167600    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df98b66d-882d-4433-a8a2-170911fd83be-hosts-file\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167620    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-lib-modules\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167665    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5rql\" (UniqueName: \"kubernetes.io/projected/2641c440-5495-414d-8912-829662eccbc4-kube-api-access-d5rql\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167674    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-device-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167689    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167680    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-bin\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167754    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-env-overrides\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc9l7\" (UniqueName: \"kubernetes.io/projected/e94daa96-3583-4bff-870d-cd25f30e49e1-kube-api-access-tc9l7\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167826    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-modprobe-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-cnibin\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.167904 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167861    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-daemon-config\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167895    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-node-log\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-socket-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-dbus\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168010    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-multus-certs\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.167524    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-netns\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168041    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-socket-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168074    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-netd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168137    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-hostroot\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-cnibin\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168192    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-socket-dir-parent\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysconfig\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168233    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/df98b66d-882d-4433-a8a2-170911fd83be-hosts-file\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.168601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168251    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-lib-modules\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168317    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-socket-dir-parent\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168367    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-modprobe-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhqz9\" (UniqueName: \"kubernetes.io/projected/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-kube-api-access-nhqz9\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168404    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysconfig\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168434    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-dbus\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168436    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-var-lib-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168476    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfj8b\" (UniqueName: \"kubernetes.io/projected/4696d571-6afb-4978-9597-86e1e0a2fe64-kube-api-access-qfj8b\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ef80e464-bdf3-47ea-b084-bd8f06a17739-konnectivity-ca\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-systemd\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-hostroot\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-system-cni-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168648    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2641c440-5495-414d-8912-829662eccbc4-system-cni-dir\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-systemd\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-systemd-units\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.169437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-tmp\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-k8s-cni-cncf-io\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168825    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-run-k8s-cni-cncf-io\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvq9n\" (UniqueName: \"kubernetes.io/projected/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-kube-api-access-jvq9n\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168974    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.168980    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169050    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-netns\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169096    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-log-socket\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169133    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-d\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169160    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-config\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-multus\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-kubelet\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169283    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-multus\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169300    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-kubelet\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-conf-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169344    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-etc-kubernetes\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-sys-fs\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-var-lib-kubelet\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169401    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-etc-kubernetes\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-sys-fs\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169410    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-kubelet-config\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169446    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-kubelet-config\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169449    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169365    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-conf-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169447    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-var-lib-kubelet\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.169529    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-registration-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169565    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169587    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-registration-dir\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.169590    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:46:48.669572152 +0000 UTC m=+3.031524571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169605    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e94daa96-3583-4bff-870d-cd25f30e49e1-etc-selinux\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb72b\" (UniqueName: \"kubernetes.io/projected/df98b66d-882d-4433-a8a2-170911fd83be-kube-api-access-nb72b\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-host\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169654    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.170973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cnibin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169682    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-kubernetes\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-run\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169706    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-host\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-systemd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169752    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-run\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-etc-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169764    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cnibin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-script-lib\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169838    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cni-binary-copy\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-kubernetes\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d32c872-6925-475a-8500-7029bc315861-ovn-node-metrics-cert\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169871    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dmsd\" (UniqueName: \"kubernetes.io/projected/804bd645-d44d-436f-afde-bc038ec57fc1-kube-api-access-9dmsd\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169952    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-multus-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169953    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/804bd645-d44d-436f-afde-bc038ec57fc1-serviceca\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.169990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4696d571-6afb-4978-9597-86e1e0a2fe64-host-slash\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.171571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170017    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-system-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170061    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df98b66d-882d-4433-a8a2-170911fd83be-tmp-dir\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170084    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-conf\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170107    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-binary-copy\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170114    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-system-cni-dir\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttq8r\" (UniqueName: \"kubernetes.io/projected/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-kube-api-access-ttq8r\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-ovn\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170205    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-bin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170223    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-sysctl-conf\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170276    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-host-var-lib-cni-bin\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170374    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/804bd645-d44d-436f-afde-bc038ec57fc1-serviceca\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170391    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170414    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-cni-binary-copy\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170561    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df98b66d-882d-4433-a8a2-170911fd83be-tmp-dir\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.170591    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2641c440-5495-414d-8912-829662eccbc4-cni-binary-copy\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.171489    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-tmp\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.171644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ef80e464-bdf3-47ea-b084-bd8f06a17739-agent-certs\") pod \"konnectivity-agent-jpwnd\" (UID: \"ef80e464-bdf3-47ea-b084-bd8f06a17739\") " pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.172114 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.171693    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-etc-tuned\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.177843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.177821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhqz9\" (UniqueName: \"kubernetes.io/projected/64ea93e6-eff0-40df-a461-9fa1ffc31d2f-kube-api-access-nhqz9\") pod \"tuned-2hjkd\" (UID: \"64ea93e6-eff0-40df-a461-9fa1ffc31d2f\") " pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.178394 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.178369    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5rql\" (UniqueName: \"kubernetes.io/projected/2641c440-5495-414d-8912-829662eccbc4-kube-api-access-d5rql\") pod \"multus-additional-cni-plugins-4m8tl\" (UID: \"2641c440-5495-414d-8912-829662eccbc4\") " pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.178492 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.178382    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc9l7\" (UniqueName: \"kubernetes.io/projected/e94daa96-3583-4bff-870d-cd25f30e49e1-kube-api-access-tc9l7\") pod \"aws-ebs-csi-driver-node-6md67\" (UID: \"e94daa96-3583-4bff-870d-cd25f30e49e1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.179718 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.179695    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:48.181504 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.181483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvq9n\" (UniqueName: \"kubernetes.io/projected/600898a9-bb9b-4b00-ac5e-ce2fef5dde22-kube-api-access-jvq9n\") pod \"multus-qm95s\" (UID: \"600898a9-bb9b-4b00-ac5e-ce2fef5dde22\") " pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.183567 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.183548    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:46:48.183667 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.183573    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:46:48.183667 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.183586    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:48.183667 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.183652    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:46:48.68363427 +0000 UTC m=+3.045586690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:48.185390 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.185370    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dmsd\" (UniqueName: \"kubernetes.io/projected/804bd645-d44d-436f-afde-bc038ec57fc1-kube-api-access-9dmsd\") pod \"node-ca-7zh6h\" (UID: \"804bd645-d44d-436f-afde-bc038ec57fc1\") " pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.186250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.186229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb72b\" (UniqueName: \"kubernetes.io/projected/df98b66d-882d-4433-a8a2-170911fd83be-kube-api-access-nb72b\") pod \"node-resolver-hnm2q\" (UID: \"df98b66d-882d-4433-a8a2-170911fd83be\") " pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.195125 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.195067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal" event={"ID":"4cd3bb05a5d106b39c55a9ac721c0cb9","Type":"ContainerStarted","Data":"964e02ee17a81e3b228683433ba071176cd169622f9033c3278f65383f236696"}
Sep 14 04:46:48.196370 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.196348    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal" event={"ID":"cb86b5cc2be15089cefdf4c423984a58","Type":"ContainerStarted","Data":"76fadb55aba6ae57bbb3a772b13a03c86cde187336d76ab787937d4f0f4ca3ae"}
Sep 14 04:46:48.270858 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttq8r\" (UniqueName: \"kubernetes.io/projected/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-kube-api-access-ttq8r\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.270858 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270865    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-ovn\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-slash\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-kubelet\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270945    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw8m5\" (UniqueName: \"kubernetes.io/projected/3d32c872-6925-475a-8500-7029bc315861-kube-api-access-cw8m5\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270966    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-ovn\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270989    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-slash\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271006    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-kubelet\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.270970    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4696d571-6afb-4978-9597-86e1e0a2fe64-iptables-alerter-script\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.271106 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271087    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271115    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-bin\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271141    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-env-overrides\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-node-log\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271171    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-bin\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271196    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-netd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-node-log\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.271195    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271246    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271279    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-var-lib-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfj8b\" (UniqueName: \"kubernetes.io/projected/4696d571-6afb-4978-9597-86e1e0a2fe64-kube-api-access-qfj8b\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271299    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-cni-netd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.271320    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:46:48.77129993 +0000 UTC m=+3.133252352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271326    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271355    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-var-lib-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.271477 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-systemd-units\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271428    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-netns\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271431    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-systemd-units\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271453    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-log-socket\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271477    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-config\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271509    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-log-socket\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-netns\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271532    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-systemd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271586    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-etc-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271589    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-host-run-ovn-kubernetes\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-script-lib\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271647    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-env-overrides\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271647    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4696d571-6afb-4978-9597-86e1e0a2fe64-iptables-alerter-script\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271662    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-run-systemd\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d32c872-6925-475a-8500-7029bc315861-ovn-node-metrics-cert\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271687    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d32c872-6925-475a-8500-7029bc315861-etc-openvswitch\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4696d571-6afb-4978-9597-86e1e0a2fe64-host-slash\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.272712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4696d571-6afb-4978-9597-86e1e0a2fe64-host-slash\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.272712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.271900    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-config\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.272712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.272069    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d32c872-6925-475a-8500-7029bc315861-ovnkube-script-lib\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.274214 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.274183    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d32c872-6925-475a-8500-7029bc315861-ovn-node-metrics-cert\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.281395 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.281340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfj8b\" (UniqueName: \"kubernetes.io/projected/4696d571-6afb-4978-9597-86e1e0a2fe64-kube-api-access-qfj8b\") pod \"iptables-alerter-kvbhv\" (UID: \"4696d571-6afb-4978-9597-86e1e0a2fe64\") " pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.281395 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.281346    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw8m5\" (UniqueName: \"kubernetes.io/projected/3d32c872-6925-475a-8500-7029bc315861-kube-api-access-cw8m5\") pod \"ovnkube-node-n7vbm\" (UID: \"3d32c872-6925-475a-8500-7029bc315861\") " pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.281395 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.281373    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttq8r\" (UniqueName: \"kubernetes.io/projected/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-kube-api-access-ttq8r\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.350525 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.350477    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2hjkd"
Sep 14 04:46:48.357999 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.357970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4m8tl"
Sep 14 04:46:48.368027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.367993    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:46:48.373010 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.372990    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67"
Sep 14 04:46:48.383684 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.383662    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qm95s"
Sep 14 04:46:48.391075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.391057    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7zh6h"
Sep 14 04:46:48.397984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.397959    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnm2q"
Sep 14 04:46:48.404767 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.404749    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kvbhv"
Sep 14 04:46:48.412432 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.412408    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:46:48.458364 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.458337    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 04:46:48.674816 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.674719    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:48.674989 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.674889    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:48.674989 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.674980    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:46:49.674958595 +0000 UTC m=+4.036911015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:48.775739 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.775704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:48.775947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:48.775753    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:48.775947 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.775876    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:48.775947 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.775901    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:46:48.775947 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.775943    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:46:48.776128 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.775956    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:46:49.775924062 +0000 UTC m=+4.137876482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:48.776128 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.775957    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:48.776128 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:48.776012    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:46:49.775995855 +0000 UTC m=+4.137948277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:49.103203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.103168    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 04:41:48 +0000 UTC" deadline="2028-02-19 11:50:19.198680727 +0000 UTC"
Sep 14 04:46:49.103203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.103198    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12559h3m30.095485184s"
Sep 14 04:46:49.112509 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.112477    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2641c440_5495_414d_8912_829662eccbc4.slice/crio-e2998e83eb56d7df130171c199594595c0932a2cf261cff43098e1d2040d2fd2 WatchSource:0}: Error finding container e2998e83eb56d7df130171c199594595c0932a2cf261cff43098e1d2040d2fd2: Status 404 returned error can't find the container with id e2998e83eb56d7df130171c199594595c0932a2cf261cff43098e1d2040d2fd2
Sep 14 04:46:49.121221 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.121192    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804bd645_d44d_436f_afde_bc038ec57fc1.slice/crio-24ee901ab9aba696cdd96467f0929068d9ec9d2c97dc03987deebc47949939a4 WatchSource:0}: Error finding container 24ee901ab9aba696cdd96467f0929068d9ec9d2c97dc03987deebc47949939a4: Status 404 returned error can't find the container with id 24ee901ab9aba696cdd96467f0929068d9ec9d2c97dc03987deebc47949939a4
Sep 14 04:46:49.122335 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.122314    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d32c872_6925_475a_8500_7029bc315861.slice/crio-68652b08351488b3d0a4cbdfc8ae1e273b8f024dc10c59916f44eb1697da7034 WatchSource:0}: Error finding container 68652b08351488b3d0a4cbdfc8ae1e273b8f024dc10c59916f44eb1697da7034: Status 404 returned error can't find the container with id 68652b08351488b3d0a4cbdfc8ae1e273b8f024dc10c59916f44eb1697da7034
Sep 14 04:46:49.124013 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.123186    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4696d571_6afb_4978_9597_86e1e0a2fe64.slice/crio-55e31e044c965df094bb1e4e34ff18021c2616efa446a7831c7b62e6b59e5584 WatchSource:0}: Error finding container 55e31e044c965df094bb1e4e34ff18021c2616efa446a7831c7b62e6b59e5584: Status 404 returned error can't find the container with id 55e31e044c965df094bb1e4e34ff18021c2616efa446a7831c7b62e6b59e5584
Sep 14 04:46:49.125597 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.125577    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94daa96_3583_4bff_870d_cd25f30e49e1.slice/crio-7a4e4bfc7993396df77d4c0bf1003ced4f2aa4956303c39cac42e26b603e3148 WatchSource:0}: Error finding container 7a4e4bfc7993396df77d4c0bf1003ced4f2aa4956303c39cac42e26b603e3148: Status 404 returned error can't find the container with id 7a4e4bfc7993396df77d4c0bf1003ced4f2aa4956303c39cac42e26b603e3148
Sep 14 04:46:49.127275 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.127208    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600898a9_bb9b_4b00_ac5e_ce2fef5dde22.slice/crio-d66e00b39583c0b4c0c1d2a06bcef3460c4b52381aacd14dce4cda53a6ad225f WatchSource:0}: Error finding container d66e00b39583c0b4c0c1d2a06bcef3460c4b52381aacd14dce4cda53a6ad225f: Status 404 returned error can't find the container with id d66e00b39583c0b4c0c1d2a06bcef3460c4b52381aacd14dce4cda53a6ad225f
Sep 14 04:46:49.128420 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:46:49.128400    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf98b66d_882d_4433_a8a2_170911fd83be.slice/crio-7eba672ad00f5b5e7309766976226f0a5a9e667762f821ad046c3c3ffc215380 WatchSource:0}: Error finding container 7eba672ad00f5b5e7309766976226f0a5a9e667762f821ad046c3c3ffc215380: Status 404 returned error can't find the container with id 7eba672ad00f5b5e7309766976226f0a5a9e667762f821ad046c3c3ffc215380
Sep 14 04:46:49.187721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.187567    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:49.187849 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.187593    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:49.187905 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.187843    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:49.187998 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.187924    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:49.200901 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.200869    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jpwnd" event={"ID":"ef80e464-bdf3-47ea-b084-bd8f06a17739","Type":"ContainerStarted","Data":"2cea68e503b6fd408372e0d7a573cdb90d25df09c0a863cd5a78670fb45e7546"}
Sep 14 04:46:49.202500 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.202472    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerStarted","Data":"e2998e83eb56d7df130171c199594595c0932a2cf261cff43098e1d2040d2fd2"}
Sep 14 04:46:49.203777 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.203751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnm2q" event={"ID":"df98b66d-882d-4433-a8a2-170911fd83be","Type":"ContainerStarted","Data":"7eba672ad00f5b5e7309766976226f0a5a9e667762f821ad046c3c3ffc215380"}
Sep 14 04:46:49.204975 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.204929    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kvbhv" event={"ID":"4696d571-6afb-4978-9597-86e1e0a2fe64","Type":"ContainerStarted","Data":"55e31e044c965df094bb1e4e34ff18021c2616efa446a7831c7b62e6b59e5584"}
Sep 14 04:46:49.206223 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.206200    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"68652b08351488b3d0a4cbdfc8ae1e273b8f024dc10c59916f44eb1697da7034"}
Sep 14 04:46:49.209491 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.209466    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zh6h" event={"ID":"804bd645-d44d-436f-afde-bc038ec57fc1","Type":"ContainerStarted","Data":"24ee901ab9aba696cdd96467f0929068d9ec9d2c97dc03987deebc47949939a4"}
Sep 14 04:46:49.210685 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.210668    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2hjkd" event={"ID":"64ea93e6-eff0-40df-a461-9fa1ffc31d2f","Type":"ContainerStarted","Data":"7e5f044af6ff5f3021d14241c5f53d57e2295543996fbb6a442a32f9f37d979d"}
Sep 14 04:46:49.216281 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.216257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qm95s" event={"ID":"600898a9-bb9b-4b00-ac5e-ce2fef5dde22","Type":"ContainerStarted","Data":"d66e00b39583c0b4c0c1d2a06bcef3460c4b52381aacd14dce4cda53a6ad225f"}
Sep 14 04:46:49.219922 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.219897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67" event={"ID":"e94daa96-3583-4bff-870d-cd25f30e49e1","Type":"ContainerStarted","Data":"7a4e4bfc7993396df77d4c0bf1003ced4f2aa4956303c39cac42e26b603e3148"}
Sep 14 04:46:49.226476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.226433    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal" podStartSLOduration=2.22641962 podStartE2EDuration="2.22641962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:46:49.226255663 +0000 UTC m=+3.588208106" watchObservedRunningTime="2026-09-14 04:46:49.22641962 +0000 UTC m=+3.588372064"
Sep 14 04:46:49.697727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.697638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:49.697947 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.697808    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:49.697947 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.697874    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:46:51.697854715 +0000 UTC m=+6.059807150 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:49.799047 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.799002    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:49.799287 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:49.799077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:49.799287 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799242    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:49.799416 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799308    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:46:51.799285138 +0000 UTC m=+6.161237574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:49.799786 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799764    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:46:49.799871 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799791    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:46:49.799871 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799803    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:49.799871 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:49.799848    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:46:51.799833468 +0000 UTC m=+6.161785904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:50.103578 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.103487    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 04:41:48 +0000 UTC" deadline="2028-03-25 21:52:35.562756845 +0000 UTC"
Sep 14 04:46:50.103578 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.103544    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13409h5m45.459216339s"
Sep 14 04:46:50.188229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.188202    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:50.188450 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:50.188306    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:50.229454 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.229410    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-105.ec2.internal" event={"ID":"4cd3bb05a5d106b39c55a9ac721c0cb9","Type":"ContainerStarted","Data":"27eef8bdc31fc7e1c669fda3b6129e9fb2a8b7f265e8fd1685514d382b4ce59f"}
Sep 14 04:46:50.244812 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.244757    2571 generic.go:356] "Generic (PLEG): container finished" podID="cb86b5cc2be15089cefdf4c423984a58" containerID="b8a7dded17ced64926fcd0500341f5a6f7b501768fcfa6c9a839dcc7e785578d" exitCode=0
Sep 14 04:46:50.245005 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:50.244823    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal" event={"ID":"cb86b5cc2be15089cefdf4c423984a58","Type":"ContainerDied","Data":"b8a7dded17ced64926fcd0500341f5a6f7b501768fcfa6c9a839dcc7e785578d"}
Sep 14 04:46:51.188216 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.188175    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:51.188735 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.188324    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:51.188735 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.188728    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:51.188860 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.188838    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:51.258978 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.258894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal" event={"ID":"cb86b5cc2be15089cefdf4c423984a58","Type":"ContainerStarted","Data":"e8cee94ff6b689ce1f49cf6164392ee044d57859a4a98d50afa969dc46ab415e"}
Sep 14 04:46:51.290150 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.286301    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-105.ec2.internal" podStartSLOduration=4.286278127 podStartE2EDuration="4.286278127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:46:51.284439425 +0000 UTC m=+5.646391871" watchObservedRunningTime="2026-09-14 04:46:51.286278127 +0000 UTC m=+5.648230573"
Sep 14 04:46:51.715129 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.715084    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:51.715796 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.715318    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:51.715796 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.715387    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:46:55.715368636 +0000 UTC m=+10.077321080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:51.816289 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.816244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:51.816459 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:51.816328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:51.816527 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.816510    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:51.816589 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.816575    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:46:55.816555669 +0000 UTC m=+10.178508093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:51.817051 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.817031    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:46:51.817125 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.817057    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:46:51.817125 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.817072    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:51.817125 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:51.817118    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:46:55.817104477 +0000 UTC m=+10.179056900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:52.190538 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:52.190445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:52.191135 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:52.190597    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:53.187791 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:53.187758    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:53.187983 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:53.187758    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:53.187983 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:53.187902    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:53.187983 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:53.187963    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:54.188418 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:54.188378    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:54.188792 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:54.188531    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:55.187813 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:55.187757    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:55.188025 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.187901    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:55.188300 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:55.188279    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:55.188424 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.188384    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:55.747095 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:55.747046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:55.747478 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.747225    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:55.747478 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.747277    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:47:03.747263704 +0000 UTC m=+18.109216128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:46:55.847601 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:55.847560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:55.847762 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:55.847622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:55.847762 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.847751    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:55.847863 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.847810    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:47:03.847790576 +0000 UTC m=+18.209743004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:46:55.848260 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.848236    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:46:55.848260 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.848257    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:46:55.848424 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.848269    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:55.848424 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:55.848311    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:47:03.848296017 +0000 UTC m=+18.210248441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:46:56.189007 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:56.188915    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:56.189161 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:56.189058    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:57.188042 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:57.188007    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:57.188042 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:57.188028    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:57.188529 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:57.188144    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:46:57.188529 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:57.188248    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:58.190189 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:58.190155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:46:58.190565 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:58.190264    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:46:59.188038 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:59.187999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:46:59.188223 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:46:59.188006    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:46:59.188223 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:59.188144    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:46:59.188223 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:46:59.188193    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:00.187884 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:00.187842    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:00.188352 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:00.188004    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:01.188162 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:01.188123    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:01.188610 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:01.188253    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:01.188610 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:01.188322    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:01.188610 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:01.188443    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:02.188084 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:02.188043    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:02.188263 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:02.188166    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:03.188610 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:03.188574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:03.189034 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:03.188623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:03.189034 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.188691    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:03.189034 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.188801    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:03.812355 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:03.812293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:03.812561 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.812488    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:47:03.812628 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.812577    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:47:19.812553882 +0000 UTC m=+34.174506320 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:47:03.912917 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:03.912872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:03.912917 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:03.912923    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913039    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913064    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913089    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913100    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913107    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:47:19.913089739 +0000 UTC m=+34.275042185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:47:03.913164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:03.913145    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:47:19.913133897 +0000 UTC m=+34.275086321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:47:04.188557 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:04.188455    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:04.188729 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:04.188608    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:05.187980 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:05.187923    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:05.187980 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:05.187983    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:05.188231 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:05.188091    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:05.188277 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:05.188227    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:06.188460 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:06.188425    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:06.188949 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:06.188529    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:07.187966 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.187674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:07.188395 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:07.188063    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:07.189107 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.188487    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:07.189107 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:07.188598    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:07.297645 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.297610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7zh6h" event={"ID":"804bd645-d44d-436f-afde-bc038ec57fc1","Type":"ContainerStarted","Data":"a17df165f560b5a6717835475343083921b9facf3fa7a5ae2897e4ce795621d7"}
Sep 14 04:47:07.299481 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.299457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67" event={"ID":"e94daa96-3583-4bff-870d-cd25f30e49e1","Type":"ContainerStarted","Data":"a3eee1ad3669c8d8d44437608b42f5477a2bb741c4d9d2c24eb760dfe8327e51"}
Sep 14 04:47:07.300769 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.300751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jpwnd" event={"ID":"ef80e464-bdf3-47ea-b084-bd8f06a17739","Type":"ContainerStarted","Data":"bafca384c8c8298003e10ee95edce9420def923d722e76be666daba015e8ee39"}
Sep 14 04:47:07.302022 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.301993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerStarted","Data":"b04996255d6b4d8f5a19d7fd42f29a0ca3bc6b0246a0e02f3f24a961c2a41242"}
Sep 14 04:47:07.334991 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.334914    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7zh6h" podStartSLOduration=3.49337228 podStartE2EDuration="21.334896426s" totalImagesPullingTime="17.841524146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:07.315455247 +0000 UTC m=+21.677407692" watchObservedRunningTime="2026-09-14 04:47:07.334896426 +0000 UTC m=+21.696848882"
Sep 14 04:47:07.429050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.428777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:07.507304 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.507272    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:07.999629 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:07.999381    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:47:08.000085 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.000066    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:47:08.049869 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.049818    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jpwnd" podStartSLOduration=4.611464767 podStartE2EDuration="22.049797504s" totalImagesPullingTime="17.438332737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:07.334599744 +0000 UTC m=+21.696552188" watchObservedRunningTime="2026-09-14 04:47:08.049797504 +0000 UTC m=+22.411749959"
Sep 14 04:47:08.188116 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.188078    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:08.188294 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:08.188234    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:08.306584 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.306545    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnm2q" event={"ID":"df98b66d-882d-4433-a8a2-170911fd83be","Type":"ContainerStarted","Data":"ff192be92c0658de1306490dca1be92f57fc8024064afe4989e164359727c4a3"}
Sep 14 04:47:08.309430 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309409    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:47:08.309717 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309696    2571 generic.go:356] "Generic (PLEG): container finished" podID="3d32c872-6925-475a-8500-7029bc315861" containerID="70b44ce29c6f409b41c3c38b637255efc83f3e27088fd98d02c24ef174fadc46" exitCode=1
Sep 14 04:47:08.309789 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309761    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"ea959a37b5424eac24b2a87ce8d5151447b8d25021ded0cea198e15aeacfa515"}
Sep 14 04:47:08.309838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309815    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"278f9d684c1de9cef5e0d533ba69903d6b0b9b5ab5c864bcdb5ad5d10bba134c"}
Sep 14 04:47:08.309888 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309847    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"e4a2f6fb6ec511c682626b1e72fdb97148c2659d0c84ea7bf7b12033747f32be"}
Sep 14 04:47:08.309888 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309882    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"06890782e9dbe1034c470023d92b6867900a7393a06081039251e30f97972c91"}
Sep 14 04:47:08.309971 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309909    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerDied","Data":"70b44ce29c6f409b41c3c38b637255efc83f3e27088fd98d02c24ef174fadc46"}
Sep 14 04:47:08.309971 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.309958    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"75191f1bac1818b1fdb1bac664d5108dce31f94826b5ba8ce5d3bbc82049fe2d"}
Sep 14 04:47:08.311183 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.311158    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2hjkd" event={"ID":"64ea93e6-eff0-40df-a461-9fa1ffc31d2f","Type":"ContainerStarted","Data":"3e060d469517b322ba54d81ad0eabd75c9325aecf1f2cf2433a0b8717ff4b631"}
Sep 14 04:47:08.312683 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.312662    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qm95s" event={"ID":"600898a9-bb9b-4b00-ac5e-ce2fef5dde22","Type":"ContainerStarted","Data":"9cc0c9b9736f4885bf5c887d57cd3b82aea64f1abc7106f06dbd5138b6070f95"}
Sep 14 04:47:08.314133 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.314110    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="b04996255d6b4d8f5a19d7fd42f29a0ca3bc6b0246a0e02f3f24a961c2a41242" exitCode=0
Sep 14 04:47:08.314198 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.314136    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"b04996255d6b4d8f5a19d7fd42f29a0ca3bc6b0246a0e02f3f24a961c2a41242"}
Sep 14 04:47:08.314370 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.314348    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:47:08.314844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.314827    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jpwnd"
Sep 14 04:47:08.333275 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.333212    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hnm2q" podStartSLOduration=4.428785954 podStartE2EDuration="22.333196548s" totalImagesPullingTime="17.904410594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:08.333131455 +0000 UTC m=+22.695083897" watchObservedRunningTime="2026-09-14 04:47:08.333196548 +0000 UTC m=+22.695148991"
Sep 14 04:47:08.386627 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.386576    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2hjkd" podStartSLOduration=4.472087096 podStartE2EDuration="22.386559882s" totalImagesPullingTime="17.914472786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:08.35961856 +0000 UTC m=+22.721571003" watchObservedRunningTime="2026-09-14 04:47:08.386559882 +0000 UTC m=+22.748512327"
Sep 14 04:47:08.492239 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.492192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:08.587370 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.587277    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:08.691277 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.691239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:08.800151 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.800108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:08.862632 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.862600    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 04:47:08.911379 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:08.911346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:09.144956 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.144760    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T04:47:08.862624305Z","UUID":"46fda114-21c2-4660-8763-a636da50e392","Handler":null,"Name":"","Endpoint":""}
Sep 14 04:47:09.146617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.146594    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 04:47:09.146773 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.146633    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 04:47:09.188378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.188352    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:09.188548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.188352    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:09.188548 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:09.188479    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:09.188635 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:09.188547    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:09.319366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.319327    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67" event={"ID":"e94daa96-3583-4bff-870d-cd25f30e49e1","Type":"ContainerStarted","Data":"ddc4e179fc8f00e58b127e87ef7109cdf8d94a080457d3ade0d69fdf4b51e2c2"}
Sep 14 04:47:09.321128 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.321099    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kvbhv" event={"ID":"4696d571-6afb-4978-9597-86e1e0a2fe64","Type":"ContainerStarted","Data":"169d9740d3bbff5bad9a8675009088cd41e49dc885ad6332f613d4f8892295e9"}
Sep 14 04:47:09.343881 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.343798    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qm95s" podStartSLOduration=5.421787808 podStartE2EDuration="23.343780303s" totalImagesPullingTime="17.921992495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:08.496168121 +0000 UTC m=+22.858120564" watchObservedRunningTime="2026-09-14 04:47:09.343780303 +0000 UTC m=+23.705732746"
Sep 14 04:47:09.501845 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.501808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:09.607093 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:09.607058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:10.188348 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.188157    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:10.188502 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:10.188475    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:10.326891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.326850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67" event={"ID":"e94daa96-3583-4bff-870d-cd25f30e49e1","Type":"ContainerStarted","Data":"df73bf51511de80a30257ccd530f12797bd427d9ec5ae1a55ae858574a6d7ad6"}
Sep 14 04:47:10.330802 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.330774    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:47:10.331357 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.331330    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"870521fef3aa8bdfb03fdaaa8065403a32ca36f5333f22a68c481e897b9ce8ee"}
Sep 14 04:47:10.349602 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.349539    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kvbhv" podStartSLOduration=6.51009698 podStartE2EDuration="24.349512475s" totalImagesPullingTime="17.839415495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:09.343171503 +0000 UTC m=+23.705123946" watchObservedRunningTime="2026-09-14 04:47:10.349512475 +0000 UTC m=+24.711464911"
Sep 14 04:47:10.350311 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:10.350271    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6md67" podStartSLOduration=3.754185675 podStartE2EDuration="24.350254594s" totalImagesPullingTime="20.596068919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:10.349684486 +0000 UTC m=+24.711636931" watchObservedRunningTime="2026-09-14 04:47:10.350254594 +0000 UTC m=+24.712207039"
Sep 14 04:47:11.017943 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:11.017903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:11.088385 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:11.088325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:11.187776 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:11.187742    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:11.187961 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:11.187745    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:11.187961 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:11.187883    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:11.188067 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:11.187988    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:12.188091 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:12.188057    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:12.188476 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:12.188182    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:13.188663 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.188482    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:13.189122 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.188478    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:13.189122 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:13.188758    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:13.189122 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:13.188813    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:13.340456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.340418    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="de8cc2238b56e4629d65762caa0306c97713636057df31ea92040974aaafc75d" exitCode=0
Sep 14 04:47:13.340617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.340509    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"de8cc2238b56e4629d65762caa0306c97713636057df31ea92040974aaafc75d"}
Sep 14 04:47:13.344287 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.344268    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:47:13.344665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.344638    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"f8e3446fa532ef597565effffb76ae2231717caf675fa0eafb3ba85d5e52f4ed"}
Sep 14 04:47:13.344943 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.344898    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:13.345051 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.344992    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:13.345132 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.345117    2571 scope.go:117] "RemoveContainer" containerID="70b44ce29c6f409b41c3c38b637255efc83f3e27088fd98d02c24ef174fadc46"
Sep 14 04:47:13.362069 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:13.362039    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:14.188704 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.188427    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:14.188704 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:14.188573    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:14.349202 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.349166    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="2b0cf5a5211c0892546142d418a317fd14b3d65fc2500b43b439e95cab2d4174" exitCode=0
Sep 14 04:47:14.349393 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.349251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"2b0cf5a5211c0892546142d418a317fd14b3d65fc2500b43b439e95cab2d4174"}
Sep 14 04:47:14.353279 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.353261    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:47:14.353624 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.353600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" event={"ID":"3d32c872-6925-475a-8500-7029bc315861","Type":"ContainerStarted","Data":"4eb0e10e7e9b15fb324bfbd67acaf8e191307565eb5b3a7d7869738b6a5cd2a9"}
Sep 14 04:47:14.353832 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.353811    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:14.369524 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.369495    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:14.414002 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.413881    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm" podStartSLOduration=10.457811869 podStartE2EDuration="28.413865722s" totalImagesPullingTime="17.956053853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:14.41213212 +0000 UTC m=+28.774084566" watchObservedRunningTime="2026-09-14 04:47:14.413865722 +0000 UTC m=+28.775818164"
Sep 14 04:47:14.485226 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.485189    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7j5q"]
Sep 14 04:47:14.485574 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.485367    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:14.485574 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:14.485484    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:14.488536 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.488501    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9nbn5"]
Sep 14 04:47:14.488694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.488655    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:14.488810 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:14.488784    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:14.489156 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.489134    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxr6b"]
Sep 14 04:47:14.489267 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:14.489250    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:14.489369 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:14.489351    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:15.359040 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:15.358762    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="5e6e09d9a740debb2d53706bd0d076ed7a94e4c16abdb3e575b46bc61dfad489" exitCode=0
Sep 14 04:47:15.359543 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:15.358849    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"5e6e09d9a740debb2d53706bd0d076ed7a94e4c16abdb3e575b46bc61dfad489"}
Sep 14 04:47:15.819650 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:15.819617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:15.908545 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:15.908512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:16.189639 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:16.189608    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:16.189796 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:16.189710    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:16.189796 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:16.189751    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:16.189796 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:16.189761    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:16.189924 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:16.189830    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:16.189924 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:16.189909    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:18.188279 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:18.188238    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:18.188726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:18.188239    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:18.188726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:18.188399    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:18.188726 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:18.188389    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:18.188726 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:18.188517    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:18.188726 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:18.188609    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:19.835282 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:19.835240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:19.835768 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.835366    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 04:47:19.835768 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.835424    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret podName:8f38c3e0-c0fb-4796-84c7-6c483ab48e2b nodeName:}" failed. No retries permitted until 2026-09-14 04:47:51.835409659 +0000 UTC m=+66.197362080 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret") pod "global-pull-secret-syncer-n7j5q" (UID: "8f38c3e0-c0fb-4796-84c7-6c483ab48e2b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 04:47:19.839830 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:19.839798    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:19.935710 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:19.935667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:19.935891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:19.935731    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:19.935891 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.935827    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 04:47:19.935891 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.935860    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 04:47:19.935891 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.935874    2571 projected.go:196] Error preparing data for projected volume kube-api-access-llwn8 for pod openshift-network-diagnostics/network-check-target-fxr6b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:47:19.935891 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.935882    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:47:19.936162 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.936001    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:47:51.935924116 +0000 UTC m=+66.297876557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 04:47:19.936162 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:19.936025    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8 podName:9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6 nodeName:}" failed. No retries permitted until 2026-09-14 04:47:51.936015092 +0000 UTC m=+66.297967514 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-llwn8" (UniqueName: "kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8") pod "network-check-target-fxr6b" (UID: "9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 04:47:19.940213 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:19.940159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:20.040489 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.040455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:20.187742 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.187699    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:20.187742 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.187721    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:20.188004 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.187710    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:20.188004 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:20.187830    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxr6b" podUID="9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6"
Sep 14 04:47:20.188004 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:20.187916    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:47:20.188149 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:20.188029    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n7j5q" podUID="8f38c3e0-c0fb-4796-84c7-6c483ab48e2b"
Sep 14 04:47:20.249044 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.249012    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:20.321565 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.321535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:20.375768 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.375688    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="ddca5b1163009def66916a11d6b9ea43d33a6b5086814a93353799920ea9e534" exitCode=0
Sep 14 04:47:20.375768 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.375745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"ddca5b1163009def66916a11d6b9ea43d33a6b5086814a93353799920ea9e534"}
Sep 14 04:47:20.959843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.959813    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-105.ec2.internal" event="NodeReady"
Sep 14 04:47:20.960250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:20.959945    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 04:47:21.009971 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.009914    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zw87b"]
Sep 14 04:47:21.033238 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.033160    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-blwf5"]
Sep 14 04:47:21.033358 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.033291    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.036008 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.035978    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 04:47:21.036154 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.035984    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 04:47:21.036154 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.036043    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wwfwc\""
Sep 14 04:47:21.057177 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.057138    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zw87b"]
Sep 14 04:47:21.057323 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.057186    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-blwf5"]
Sep 14 04:47:21.057323 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.057273    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.060871 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.060848    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 04:47:21.061370 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.061351    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 04:47:21.061464 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.061378    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-46brg\""
Sep 14 04:47:21.061663 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.061644    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 04:47:21.144431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-cert\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.144431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3bebbe5-347c-46df-b90e-542a9c8882db-config-volume\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.144646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144461    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxl8m\" (UniqueName: \"kubernetes.io/projected/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-kube-api-access-vxl8m\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.144646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144579    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.144646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144611    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcpts\" (UniqueName: \"kubernetes.io/projected/f3bebbe5-347c-46df-b90e-542a9c8882db-kube-api-access-qcpts\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.144646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.144631    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bebbe5-347c-46df-b90e-542a9c8882db-tmp-dir\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.186090 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.186062    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:21.245286 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245246    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxl8m\" (UniqueName: \"kubernetes.io/projected/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-kube-api-access-vxl8m\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.245494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.245494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245325    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcpts\" (UniqueName: \"kubernetes.io/projected/f3bebbe5-347c-46df-b90e-542a9c8882db-kube-api-access-qcpts\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.245494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bebbe5-347c-46df-b90e-542a9c8882db-tmp-dir\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.245494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245382    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-cert\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.245494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3bebbe5-347c-46df-b90e-542a9c8882db-config-volume\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.245739 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:21.245519    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:21.245739 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:21.245590    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:21.745571664 +0000 UTC m=+36.107524086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:21.245833 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245742    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bebbe5-347c-46df-b90e-542a9c8882db-tmp-dir\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.245987 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.245971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3bebbe5-347c-46df-b90e-542a9c8882db-config-volume\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.249988 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.249962    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-cert\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.255502 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.255477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxl8m\" (UniqueName: \"kubernetes.io/projected/fc32cf52-6d5e-489a-9589-3cc91fe9dbd2-kube-api-access-vxl8m\") pod \"ingress-canary-blwf5\" (UID: \"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2\") " pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.255618 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.255539    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcpts\" (UniqueName: \"kubernetes.io/projected/f3bebbe5-347c-46df-b90e-542a9c8882db-kube-api-access-qcpts\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.290251 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.290170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:21.365750 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.365710    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-blwf5"
Sep 14 04:47:21.382071 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.382039    2571 generic.go:356] "Generic (PLEG): container finished" podID="2641c440-5495-414d-8912-829662eccbc4" containerID="7db2bfc5e3abf234e8b2f367c917a9c2459daabb5322472d50ae81dae254f914" exitCode=0
Sep 14 04:47:21.382253 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.382085    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerDied","Data":"7db2bfc5e3abf234e8b2f367c917a9c2459daabb5322472d50ae81dae254f914"}
Sep 14 04:47:21.686417 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.686378    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-blwf5"]
Sep 14 04:47:21.691117 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:47:21.691077    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc32cf52_6d5e_489a_9589_3cc91fe9dbd2.slice/crio-c28323751de39322bbbb249585e116828474bcfda606cd56a5108afa41224c61 WatchSource:0}: Error finding container c28323751de39322bbbb249585e116828474bcfda606cd56a5108afa41224c61: Status 404 returned error can't find the container with id c28323751de39322bbbb249585e116828474bcfda606cd56a5108afa41224c61
Sep 14 04:47:21.749953 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:21.749896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:21.750123 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:21.750054    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:21.750123 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:21.750119    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:22.75010405 +0000 UTC m=+37.112056471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:22.188032 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.187999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:22.188032 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.188014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:22.188676 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.188255    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:22.191271 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.191248    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kgkvn\""
Sep 14 04:47:22.192328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.192312    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 04:47:22.192444 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.192328    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 04:47:22.192444 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.192337    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 04:47:22.192444 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.192387    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 04:47:22.194841 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.194820    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-54hj4\""
Sep 14 04:47:22.386643 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.386603    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-blwf5" event={"ID":"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2","Type":"ContainerStarted","Data":"c28323751de39322bbbb249585e116828474bcfda606cd56a5108afa41224c61"}
Sep 14 04:47:22.390694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.390665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" event={"ID":"2641c440-5495-414d-8912-829662eccbc4","Type":"ContainerStarted","Data":"c5bc76c117586ef06691d9a2596c7bc4060a4df57d5240f9344527034eac0c72"}
Sep 14 04:47:22.419377 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.419322    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4m8tl" podStartSLOduration=8.265383942 podStartE2EDuration="36.419298351s" totalImagesPullingTime="28.153914409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:22.418739729 +0000 UTC m=+36.780692173" watchObservedRunningTime="2026-09-14 04:47:22.419298351 +0000 UTC m=+36.781250793"
Sep 14 04:47:22.550126 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.550089    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:22.653446 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.653415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:22.759436 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:22.759401    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:22.759626 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:22.759535    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:22.759626 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:22.759599    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:24.759579146 +0000 UTC m=+39.121531568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:24.776366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:24.776181    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:24.776366 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:24.776349    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:24.776977 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:24.776432    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:28.776411968 +0000 UTC m=+43.138364393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:25.228675 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:25.228643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:25.296639 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:25.296605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:25.399946 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:25.399906    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-blwf5" event={"ID":"fc32cf52-6d5e-489a-9589-3cc91fe9dbd2","Type":"ContainerStarted","Data":"dda83fe14eddf87ae98df8f80b890be437f9ecc503d8c85de22545b75ca7ff22"}
Sep 14 04:47:25.418677 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:25.418619    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-blwf5" podStartSLOduration=2.802218304 podStartE2EDuration="5.418600399s" totalImagesPullingTime="2.616382095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:25.418238731 +0000 UTC m=+39.780191177" watchObservedRunningTime="2026-09-14 04:47:25.418600399 +0000 UTC m=+39.780552843"
Sep 14 04:47:28.803616 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:28.803559    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:28.804164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:28.803729    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:28.804164 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:28.803824    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:36.803803877 +0000 UTC m=+51.165756317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:28.943085 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:28.943051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:29.043641 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:29.043609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:36.856104 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:36.856063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:36.856661 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:36.856251    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:36.856661 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:36.856332    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:47:52.856309885 +0000 UTC m=+67.218262309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:41.842412 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:41.842372    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:41.907322 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:41.907287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:46.372987 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:46.372956    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n7vbm"
Sep 14 04:47:46.545952 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:46.545903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:46.666266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:46.666182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:47:51.857516 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.857451    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:51.861109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.861089    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 04:47:51.871493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.871463    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8f38c3e0-c0fb-4796-84c7-6c483ab48e2b-original-pull-secret\") pod \"global-pull-secret-syncer-n7j5q\" (UID: \"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b\") " pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:51.905778 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.905738    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n7j5q"
Sep 14 04:47:51.958818 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.958776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:47:51.958966 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.958891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:51.962205 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.962187    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 04:47:51.963027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.963008    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 04:47:51.969538 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:51.969507    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 04:47:51.969785 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:51.969763    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:48:55.969738031 +0000 UTC m=+130.331690455 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : secret "metrics-daemon-secret" not found
Sep 14 04:47:51.972969 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.972923    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 04:47:51.982593 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:51.982564    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llwn8\" (UniqueName: \"kubernetes.io/projected/9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6-kube-api-access-llwn8\") pod \"network-check-target-fxr6b\" (UID: \"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6\") " pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:52.177379 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.177296    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n7j5q"]
Sep 14 04:47:52.181289 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:47:52.181261    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f38c3e0_c0fb_4796_84c7_6c483ab48e2b.slice/crio-a9b246d9933c6124d5f611edf1504fd53f4c0ac2b880a239319afcf89c8654d3 WatchSource:0}: Error finding container a9b246d9933c6124d5f611edf1504fd53f4c0ac2b880a239319afcf89c8654d3: Status 404 returned error can't find the container with id a9b246d9933c6124d5f611edf1504fd53f4c0ac2b880a239319afcf89c8654d3
Sep 14 04:47:52.202020 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.201981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-54hj4\""
Sep 14 04:47:52.209489 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.209461    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:52.465726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.465689    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7j5q" event={"ID":"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b","Type":"ContainerStarted","Data":"a9b246d9933c6124d5f611edf1504fd53f4c0ac2b880a239319afcf89c8654d3"}
Sep 14 04:47:52.483043 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.483009    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxr6b"]
Sep 14 04:47:52.486114 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:47:52.486089    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a85b9fb_3bd0_4d93_ad30_c2c7ba335cf6.slice/crio-46d4eb1b2ce6d2a0d5d67d3c6da44a6f20786d77e293f6c271b9c08f5f221496 WatchSource:0}: Error finding container 46d4eb1b2ce6d2a0d5d67d3c6da44a6f20786d77e293f6c271b9c08f5f221496: Status 404 returned error can't find the container with id 46d4eb1b2ce6d2a0d5d67d3c6da44a6f20786d77e293f6c271b9c08f5f221496
Sep 14 04:47:52.866761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:52.866665    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:47:52.867227 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:52.866831    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:47:52.867227 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:47:52.866912    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:48:24.866889951 +0000 UTC m=+99.228842374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:47:53.470672 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:53.470629    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxr6b" event={"ID":"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6","Type":"ContainerStarted","Data":"46d4eb1b2ce6d2a0d5d67d3c6da44a6f20786d77e293f6c271b9c08f5f221496"}
Sep 14 04:47:57.483169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:57.483123    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n7j5q" event={"ID":"8f38c3e0-c0fb-4796-84c7-6c483ab48e2b","Type":"ContainerStarted","Data":"3b6ce01325a0ce2d6d7410f462d9236e5b5b0514fd653bc83078987341a7b8dd"}
Sep 14 04:47:57.484635 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:57.484611    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxr6b" event={"ID":"9a85b9fb-3bd0-4d93-ad30-c2c7ba335cf6","Type":"ContainerStarted","Data":"8333cc98f6cb485baaf40b012f9d9d0428781178564bc2d04b9b274e15ba4b02"}
Sep 14 04:47:57.484765 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:57.484747    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:47:57.501324 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:57.501256    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n7j5q" podStartSLOduration=66.73780484 podStartE2EDuration="1m11.50123471s" totalImagesPullingTime="4.76342987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:57.501038017 +0000 UTC m=+71.862990460" watchObservedRunningTime="2026-09-14 04:47:57.50123471 +0000 UTC m=+71.863187151"
Sep 14 04:47:57.521280 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:47:57.521228    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fxr6b" podStartSLOduration=67.066864229 podStartE2EDuration="1m11.521209702s" totalImagesPullingTime="4.454345473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:47:57.520405764 +0000 UTC m=+71.882358208" watchObservedRunningTime="2026-09-14 04:47:57.521209702 +0000 UTC m=+71.883162145"
Sep 14 04:48:24.892583 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:24.892526    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:48:24.893054 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:24.892674    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 04:48:24.893054 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:24.892741    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls podName:f3bebbe5-347c-46df-b90e-542a9c8882db nodeName:}" failed. No retries permitted until 2026-09-14 04:49:28.892726146 +0000 UTC m=+163.254678567 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls") pod "dns-default-zw87b" (UID: "f3bebbe5-347c-46df-b90e-542a9c8882db") : secret "dns-default-metrics-tls" not found
Sep 14 04:48:28.490607 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:28.490567    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fxr6b"
Sep 14 04:48:40.892984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.892947    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"]
Sep 14 04:48:40.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.895547    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dn8r4"]
Sep 14 04:48:40.895726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.895700    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:40.898550 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.898523    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:40.900642 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.899334    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:48:40.901040 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.899419    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 14 04:48:40.901665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.899461    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 14 04:48:40.901816 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.899487    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tcmq4\""
Sep 14 04:48:40.901967 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.899521    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 14 04:48:40.903330 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.903305    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 14 04:48:40.903330 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.903305    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 14 04:48:40.903605 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.903591    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 14 04:48:40.904098 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.904079    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:48:40.904248 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.904235    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pj8ql\""
Sep 14 04:48:40.908444 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.908418    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"]
Sep 14 04:48:40.909276 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.909255    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dn8r4"]
Sep 14 04:48:40.909448 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:40.909431    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 14 04:48:41.001587 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-config\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.001587 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001593    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.001838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001616    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.001838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-kube-api-access-tbnnn\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.001838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-serving-cert\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.001838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sw5m\" (UniqueName: \"kubernetes.io/projected/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-kube-api-access-8sw5m\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.002056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.001908    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-trusted-ca\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.093473 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.093438    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"]
Sep 14 04:48:41.096313 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.096289    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.098890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.098851    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 14 04:48:41.098890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.098872    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 14 04:48:41.099083 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.098873    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r7rgk\""
Sep 14 04:48:41.099165 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.099151    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:48:41.102420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102397    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:48:41.102564 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-serving-cert\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.102622 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102584    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sw5m\" (UniqueName: \"kubernetes.io/projected/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-kube-api-access-8sw5m\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.102698 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-trusted-ca\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.102746 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-config\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.102795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102747    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.102795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102773    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.102905 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.102799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-kube-api-access-tbnnn\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.103677 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.103533    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-config\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.103865 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.103840    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.104024 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.104006    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-trusted-ca\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.105549 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.105530    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.105667 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.105650    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.105737 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.105710    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-serving-cert\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.108224 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.108205    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 04:48:41.108325 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.108246    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 04:48:41.108325 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.108274    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgrqm\""
Sep 14 04:48:41.108528 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.108515    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 04:48:41.113695 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.113669    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 04:48:41.116233 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.116211    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"]
Sep 14 04:48:41.118857 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.118635    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sw5m\" (UniqueName: \"kubernetes.io/projected/c016e1b5-cc4b-41f5-abe7-424d6a8023d7-kube-api-access-8sw5m\") pod \"console-operator-696884fb9c-dn8r4\" (UID: \"c016e1b5-cc4b-41f5-abe7-424d6a8023d7\") " pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.120644 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.120619    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:48:41.122748 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.122720    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnnn\" (UniqueName: \"kubernetes.io/projected/27a1027d-4dc2-4f57-a251-57eb0b9e95b3-kube-api-access-tbnnn\") pod \"kube-storage-version-migrator-operator-6c8f84f897-fsrgh\" (UID: \"27a1027d-4dc2-4f57-a251-57eb0b9e95b3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.203664 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203671    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203723    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203850    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.203899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203873    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2k25\" (UniqueName: \"kubernetes.io/projected/1ec5d81a-09a2-489c-a520-a434d71fc031-kube-api-access-t2k25\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.204181 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203901    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.204181 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203925    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.204181 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.203972    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7qsz\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.208632 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.208613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"
Sep 14 04:48:41.214520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.214493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:41.284175 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.284089    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:41.305142 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305107    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305221    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2k25\" (UniqueName: \"kubernetes.io/projected/1ec5d81a-09a2-489c-a520-a434d71fc031-kube-api-access-t2k25\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305245    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7qsz\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305596 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.305341    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 04:48:41.305596 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305596 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.305416    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls podName:1ec5d81a-09a2-489c-a520-a434d71fc031 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:41.805394607 +0000 UTC m=+116.167347049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-dz8zn" (UID: "1ec5d81a-09a2-489c-a520-a434d71fc031") : secret "samples-operator-tls" not found
Sep 14 04:48:41.305596 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305441    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305596 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305485    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305890 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.305612    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 04:48:41.305890 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.305625    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6894f7db9d-64t5p: secret "image-registry-tls" not found
Sep 14 04:48:41.305890 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.305670    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls podName:f0add8d8-f38a-4337-842c-d1824f1e521c nodeName:}" failed. No retries permitted until 2026-09-14 04:48:41.805648283 +0000 UTC m=+116.167600711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls") pod "image-registry-6894f7db9d-64t5p" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c") : secret "image-registry-tls" not found
Sep 14 04:48:41.305890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305666    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.305890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.305764    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.306531 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.306512    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.307967 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.307921    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.308054 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.307947    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.315199 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.315173    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.315294 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.315203    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7qsz\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.316948 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.316902    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2k25\" (UniqueName: \"kubernetes.io/projected/1ec5d81a-09a2-489c-a520-a434d71fc031-kube-api-access-t2k25\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.492673 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.492565    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-dn8r4"]
Sep 14 04:48:41.500021 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.498588    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh"]
Sep 14 04:48:41.500564 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:48:41.500536    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a1027d_4dc2_4f57_a251_57eb0b9e95b3.slice/crio-eae636780dcc8c9efeb34c8105e561efc43a6577dd02f92b3309a904fa287b95 WatchSource:0}: Error finding container eae636780dcc8c9efeb34c8105e561efc43a6577dd02f92b3309a904fa287b95: Status 404 returned error can't find the container with id eae636780dcc8c9efeb34c8105e561efc43a6577dd02f92b3309a904fa287b95
Sep 14 04:48:41.596689 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.596653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh" event={"ID":"27a1027d-4dc2-4f57-a251-57eb0b9e95b3","Type":"ContainerStarted","Data":"eae636780dcc8c9efeb34c8105e561efc43a6577dd02f92b3309a904fa287b95"}
Sep 14 04:48:41.597723 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.597701    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" event={"ID":"c016e1b5-cc4b-41f5-abe7-424d6a8023d7","Type":"ContainerStarted","Data":"e2991607ade8599c6ac836fd297dbe5f82f704739dd58dd8c6ed1d52cd9d546d"}
Sep 14 04:48:41.807761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.807655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:41.807761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:41.807729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:41.808021 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.807815    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 04:48:41.808021 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.807830    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6894f7db9d-64t5p: secret "image-registry-tls" not found
Sep 14 04:48:41.808021 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.807869    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 04:48:41.808021 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.807884    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls podName:f0add8d8-f38a-4337-842c-d1824f1e521c nodeName:}" failed. No retries permitted until 2026-09-14 04:48:42.807868054 +0000 UTC m=+117.169820474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls") pod "image-registry-6894f7db9d-64t5p" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c") : secret "image-registry-tls" not found
Sep 14 04:48:41.808021 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:41.807921    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls podName:1ec5d81a-09a2-489c-a520-a434d71fc031 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:42.807907078 +0000 UTC m=+117.169859498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-dz8zn" (UID: "1ec5d81a-09a2-489c-a520-a434d71fc031") : secret "samples-operator-tls" not found
Sep 14 04:48:42.747350 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:42.747302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:42.816888 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:42.816842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:42.817073 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:42.816959    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:42.817073 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:42.816991    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 04:48:42.817073 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:42.817011    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6894f7db9d-64t5p: secret "image-registry-tls" not found
Sep 14 04:48:42.817073 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:42.817074    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls podName:f0add8d8-f38a-4337-842c-d1824f1e521c nodeName:}" failed. No retries permitted until 2026-09-14 04:48:44.817058089 +0000 UTC m=+119.179010510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls") pod "image-registry-6894f7db9d-64t5p" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c") : secret "image-registry-tls" not found
Sep 14 04:48:42.817228 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:42.817090    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 04:48:42.817228 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:42.817139    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls podName:1ec5d81a-09a2-489c-a520-a434d71fc031 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:44.817126674 +0000 UTC m=+119.179079095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-dz8zn" (UID: "1ec5d81a-09a2-489c-a520-a434d71fc031") : secret "samples-operator-tls" not found
Sep 14 04:48:43.182044 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:43.181954    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:43.255901 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:43.255861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:44.607236 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.607194    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh" event={"ID":"27a1027d-4dc2-4f57-a251-57eb0b9e95b3","Type":"ContainerStarted","Data":"c6ea17ab59d9eb3e6e303b102fa3a7cef4103d105c434209cc0aaff72710f8ca"}
Sep 14 04:48:44.608857 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.608835    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/0.log"
Sep 14 04:48:44.608998 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.608874    2571 generic.go:356] "Generic (PLEG): container finished" podID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7" containerID="1c81a0da4abc08eae957fa48f567f58920742aaf101ce658b30b7b0ba3243019" exitCode=255
Sep 14 04:48:44.608998 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.608904    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" event={"ID":"c016e1b5-cc4b-41f5-abe7-424d6a8023d7","Type":"ContainerDied","Data":"1c81a0da4abc08eae957fa48f567f58920742aaf101ce658b30b7b0ba3243019"}
Sep 14 04:48:44.609141 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.609127    2571 scope.go:117] "RemoveContainer" containerID="1c81a0da4abc08eae957fa48f567f58920742aaf101ce658b30b7b0ba3243019"
Sep 14 04:48:44.631542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.631498    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh" podStartSLOduration=2.302186294 podStartE2EDuration="4.631481788s" totalImagesPullingTime="2.329295494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:48:44.631096276 +0000 UTC m=+118.993048717" watchObservedRunningTime="2026-09-14 04:48:44.631481788 +0000 UTC m=+118.993434232"
Sep 14 04:48:44.835890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.835848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:44.836083 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:44.835918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:44.836083 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:44.836029    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 04:48:44.836083 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:44.836033    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 04:48:44.836083 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:44.836053    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6894f7db9d-64t5p: secret "image-registry-tls" not found
Sep 14 04:48:44.836259 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:44.836094    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls podName:1ec5d81a-09a2-489c-a520-a434d71fc031 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:48.836078564 +0000 UTC m=+123.198030985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-dz8zn" (UID: "1ec5d81a-09a2-489c-a520-a434d71fc031") : secret "samples-operator-tls" not found
Sep 14 04:48:44.836259 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:44.836107    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls podName:f0add8d8-f38a-4337-842c-d1824f1e521c nodeName:}" failed. No retries permitted until 2026-09-14 04:48:48.836101192 +0000 UTC m=+123.198053612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls") pod "image-registry-6894f7db9d-64t5p" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c") : secret "image-registry-tls" not found
Sep 14 04:48:45.027700 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.027666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:45.057573 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.055696    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-55sbp"]
Sep 14 04:48:45.059792 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.059763    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.066056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.065173    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 04:48:45.066056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.065530    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d8h7x\""
Sep 14 04:48:45.066056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.065714    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 04:48:45.066056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.065866    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 04:48:45.066382 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.066122    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 04:48:45.078234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.078198    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-55sbp"]
Sep 14 04:48:45.127246 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.127205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:45.138382 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.138341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.138576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.138424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.138576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.138475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5369a29b-707b-42ed-a29c-f34c71fce8f8-data-volume\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.138576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.138499    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npnq7\" (UniqueName: \"kubernetes.io/projected/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-api-access-npnq7\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.138576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.138555    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5369a29b-707b-42ed-a29c-f34c71fce8f8-crio-socket\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.239906 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.239861    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5369a29b-707b-42ed-a29c-f34c71fce8f8-crio-socket\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.239906 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.239914    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.240012    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.240030    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5369a29b-707b-42ed-a29c-f34c71fce8f8-crio-socket\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.240068    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:45.740053559 +0000 UTC m=+120.102005979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : configmap "kube-rbac-proxy" not found
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.240116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.240186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5369a29b-707b-42ed-a29c-f34c71fce8f8-data-volume\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.240193 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.240198    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:45.240431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.240217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npnq7\" (UniqueName: \"kubernetes.io/projected/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-api-access-npnq7\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.240431 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.240223    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:45.740215395 +0000 UTC m=+120.102167816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:45.240496 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.240474    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5369a29b-707b-42ed-a29c-f34c71fce8f8-data-volume\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.249970 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.249922    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npnq7\" (UniqueName: \"kubernetes.io/projected/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-api-access-npnq7\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.446778 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.446685    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:45.512324 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.512291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:45.614340 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.614311    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/1.log"
Sep 14 04:48:45.614732 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.614722    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/0.log"
Sep 14 04:48:45.614776 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.614754    2571 generic.go:356] "Generic (PLEG): container finished" podID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4" exitCode=255
Sep 14 04:48:45.614895 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.614853    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" event={"ID":"c016e1b5-cc4b-41f5-abe7-424d6a8023d7","Type":"ContainerDied","Data":"97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"}
Sep 14 04:48:45.614985 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.614917    2571 scope.go:117] "RemoveContainer" containerID="1c81a0da4abc08eae957fa48f567f58920742aaf101ce658b30b7b0ba3243019"
Sep 14 04:48:45.615340 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.615166    2571 scope.go:117] "RemoveContainer" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"
Sep 14 04:48:45.615481 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.615366    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dn8r4_openshift-console-operator(c016e1b5-cc4b-41f5-abe7-424d6a8023d7)\"" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podUID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7"
Sep 14 04:48:45.743420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.743381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.743599 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:45.743523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:45.743599 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.743544    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:45.743721 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.743621    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:46.743600332 +0000 UTC m=+121.105552776 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:45.743721 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.743642    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 04:48:45.743721 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:45.743702    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:46.743684112 +0000 UTC m=+121.105636552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : configmap "kube-rbac-proxy" not found
Sep 14 04:48:46.090793 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.090713    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:46.194097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.194061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:46.296086 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.296052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:46.619899 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.619870    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/1.log"
Sep 14 04:48:46.620302 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.620216    2571 scope.go:117] "RemoveContainer" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"
Sep 14 04:48:46.620387 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:46.620368    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dn8r4_openshift-console-operator(c016e1b5-cc4b-41f5-abe7-424d6a8023d7)\"" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podUID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7"
Sep 14 04:48:46.752346 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.752295    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:46.752558 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.752398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:46.752558 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:46.752423    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 04:48:46.752558 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:46.752487    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:48.752469925 +0000 UTC m=+123.114422346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : configmap "kube-rbac-proxy" not found
Sep 14 04:48:46.752759 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:46.752575    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:46.752759 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:46.752628    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:48.75261296 +0000 UTC m=+123.114565381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:46.774596 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:46.774567    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnm2q_df98b66d-882d-4433-a8a2-170911fd83be/dns-node-resolver/0.log"
Sep 14 04:48:47.871967 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:47.871921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:47.988111 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:47.988082    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zh6h_804bd645-d44d-436f-afde-bc038ec57fc1/node-ca/0.log"
Sep 14 04:48:48.034802 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.034771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:48.100353 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.100324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:48.766628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.766592    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:48.766846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.766709    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:48.766846 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.766754    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:48.766846 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.766799    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 04:48:48.766846 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.766822    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:52.766804178 +0000 UTC m=+127.128756602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:48.767055 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.766851    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:52.766835086 +0000 UTC m=+127.128787521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : configmap "kube-rbac-proxy" not found
Sep 14 04:48:48.776588 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.776563    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-blwf5_fc32cf52-6d5e-489a-9589-3cc91fe9dbd2/serve-healthcheck-canary/0.log"
Sep 14 04:48:48.867561 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.867525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:48.867747 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:48.867598    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:48.867747 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.867675    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 04:48:48.867747 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.867696    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6894f7db9d-64t5p: secret "image-registry-tls" not found
Sep 14 04:48:48.867747 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.867698    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 04:48:48.867747 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.867752    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls podName:1ec5d81a-09a2-489c-a520-a434d71fc031 nodeName:}" failed. No retries permitted until 2026-09-14 04:48:56.867734597 +0000 UTC m=+131.229687019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-dz8zn" (UID: "1ec5d81a-09a2-489c-a520-a434d71fc031") : secret "samples-operator-tls" not found
Sep 14 04:48:48.867967 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:48.867770    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls podName:f0add8d8-f38a-4337-842c-d1824f1e521c nodeName:}" failed. No retries permitted until 2026-09-14 04:48:56.867760677 +0000 UTC m=+131.229713106 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls") pod "image-registry-6894f7db9d-64t5p" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c") : secret "image-registry-tls" not found
Sep 14 04:48:49.458820 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:49.458786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:51.215211 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.215167    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:51.215605 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.215239    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:48:51.215605 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.215505    2571 scope.go:117] "RemoveContainer" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"
Sep 14 04:48:51.215690 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:51.215668    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-dn8r4_openshift-console-operator(c016e1b5-cc4b-41f5-abe7-424d6a8023d7)\"" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podUID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7"
Sep 14 04:48:51.660547 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.660463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:51.818535 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.818501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:51.921450 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:51.921363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:52.031665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.031631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:52.131507 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.131469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:52.249604 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.249572    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:52.351506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.351477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:52.803312 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.803268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:52.803487 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.803332    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:48:52.803487 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:52.803386    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 04:48:52.803487 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:52.803449    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:52.803487 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:52.803468    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:49:00.80344404 +0000 UTC m=+135.165396463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : configmap "kube-rbac-proxy" not found
Sep 14 04:48:52.803641 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:52.803501    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:49:00.803488678 +0000 UTC m=+135.165441099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:48:52.864467 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:52.864425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:53.537643 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:53.537617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:53.738635 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:53.738601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:53.935959 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:53.935859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:54.169123 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:54.169086    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:54.343367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:54.343332    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:54.934952 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:54.934882    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:55.330343 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:55.330304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:55.743454 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:55.743423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:55.844138 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:55.844108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:55.944890 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:55.944855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:56.029369 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.029292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:48:56.029479 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:56.029412    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 04:48:56.029479 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:48:56.029468    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs podName:f90d5ac3-fe7e-4f0f-83bf-33aa002548bc nodeName:}" failed. No retries permitted until 2026-09-14 04:50:58.029454747 +0000 UTC m=+252.391407168 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs") pod "network-metrics-daemon-9nbn5" (UID: "f90d5ac3-fe7e-4f0f-83bf-33aa002548bc") : secret "metrics-daemon-secret" not found
Sep 14 04:48:56.129611 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.129568    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:56.326497 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.326410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:56.388803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.388759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:56.936120 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.936077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:56.936304 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.936132    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:56.938663 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.938630    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"image-registry-6894f7db9d-64t5p\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") " pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:56.938663 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:56.938658    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ec5d81a-09a2-489c-a520-a434d71fc031-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-dz8zn\" (UID: \"1ec5d81a-09a2-489c-a520-a434d71fc031\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:57.009732 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.009700    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r7rgk\""
Sep 14 04:48:57.017883 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.017861    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"
Sep 14 04:48:57.035507 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.035473    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xgrqm\""
Sep 14 04:48:57.043108 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.043076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:57.292357 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.292323    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn"]
Sep 14 04:48:57.324557 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.324524    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:48:57.327403 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:48:57.327375    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0add8d8_f38a_4337_842c_d1824f1e521c.slice/crio-e6290e254f26559caf4efca80f1f63426e1f269c597d4b89d5a56ec43473973d WatchSource:0}: Error finding container e6290e254f26559caf4efca80f1f63426e1f269c597d4b89d5a56ec43473973d: Status 404 returned error can't find the container with id e6290e254f26559caf4efca80f1f63426e1f269c597d4b89d5a56ec43473973d
Sep 14 04:48:57.533687 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.533650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:57.656686 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.656589    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn" event={"ID":"1ec5d81a-09a2-489c-a520-a434d71fc031","Type":"ContainerStarted","Data":"b0868ed8042a5c5ca76df8231f84b0e8c4d03047d0df65653aad5abbbf31eec3"}
Sep 14 04:48:57.658255 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.658227    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" event={"ID":"f0add8d8-f38a-4337-842c-d1824f1e521c","Type":"ContainerStarted","Data":"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"}
Sep 14 04:48:57.658416 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.658268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" event={"ID":"f0add8d8-f38a-4337-842c-d1824f1e521c","Type":"ContainerStarted","Data":"e6290e254f26559caf4efca80f1f63426e1f269c597d4b89d5a56ec43473973d"}
Sep 14 04:48:57.658416 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.658383    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:48:57.679084 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.679015    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" podStartSLOduration=16.678992259 podStartE2EDuration="16.678992259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:48:57.67857991 +0000 UTC m=+132.040532378" watchObservedRunningTime="2026-09-14 04:48:57.678992259 +0000 UTC m=+132.040944700"
Sep 14 04:48:57.730514 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.730482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:57.913697 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.913607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:57.977610 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:57.977565    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:58.129295 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:58.129260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:58.229767 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:58.229730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:58.336984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:58.336948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:59.331490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.331400    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:59.666220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.666125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn" event={"ID":"1ec5d81a-09a2-489c-a520-a434d71fc031","Type":"ContainerStarted","Data":"137aca67aaabff82f80cb08bf516a0d0b770c50370ac3039776ec3d60ecdef3b"}
Sep 14 04:48:59.666220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.666174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn" event={"ID":"1ec5d81a-09a2-489c-a520-a434d71fc031","Type":"ContainerStarted","Data":"b3e1229b45e48d1ff47321b008674dabaa0ef4d84aa23a573222490023f99b46"}
Sep 14 04:48:59.688154 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.688092    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-dz8zn" podStartSLOduration=17.041042487 podStartE2EDuration="18.688072512s" totalImagesPullingTime="1.647030025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:48:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:48:59.686395827 +0000 UTC m=+134.048348271" watchObservedRunningTime="2026-09-14 04:48:59.688072512 +0000 UTC m=+134.050024955"
Sep 14 04:48:59.753616 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.753579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:48:59.937708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:48:59.937615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:00.131957 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.131901    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:00.333255 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.333224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:00.666682 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.666602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:00.867220 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.867176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:00.867391 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.867236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:00.867391 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:00.867342    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 04:49:00.867472 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:00.867395    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls podName:5369a29b-707b-42ed-a29c-f34c71fce8f8 nodeName:}" failed. No retries permitted until 2026-09-14 04:49:16.867380594 +0000 UTC m=+151.229333014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-55sbp" (UID: "5369a29b-707b-42ed-a29c-f34c71fce8f8") : secret "insights-runtime-extractor-tls" not found
Sep 14 04:49:00.867740 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:00.867718    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5369a29b-707b-42ed-a29c-f34c71fce8f8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:01.335577 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:01.335544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:01.731428 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:01.731397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:01.929777 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:01.929746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:02.137036 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:02.136951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:02.344467 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:02.344435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:04.188301 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.188265    2571 scope.go:117] "RemoveContainer" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"
Sep 14 04:49:04.681286 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.681255    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:49:04.681681 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.681666    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/1.log"
Sep 14 04:49:04.681745 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.681698    2571 generic.go:356] "Generic (PLEG): container finished" podID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7" containerID="c6909e957690e5218e274b6610e75d727d753f667b97bed87e7d1dbad2db284c" exitCode=255
Sep 14 04:49:04.681779 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.681747    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" event={"ID":"c016e1b5-cc4b-41f5-abe7-424d6a8023d7","Type":"ContainerDied","Data":"c6909e957690e5218e274b6610e75d727d753f667b97bed87e7d1dbad2db284c"}
Sep 14 04:49:04.681821 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.681792    2571 scope.go:117] "RemoveContainer" containerID="97621efc0749a32702698b373dfe843f2a3792366a4afe6ef38fc350a2f499c4"
Sep 14 04:49:04.682129 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:04.682104    2571 scope.go:117] "RemoveContainer" containerID="c6909e957690e5218e274b6610e75d727d753f667b97bed87e7d1dbad2db284c"
Sep 14 04:49:04.682292 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:04.682271    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-dn8r4_openshift-console-operator(c016e1b5-cc4b-41f5-abe7-424d6a8023d7)\"" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podUID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7"
Sep 14 04:49:05.686571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:05.686542    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:49:09.775185 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.775143    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"]
Sep 14 04:49:09.778225 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.778205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:09.782257 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.782230    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 04:49:09.782427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.782256    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-gc9ct\""
Sep 14 04:49:09.782427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.782257    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 04:49:09.787315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.787279    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"]
Sep 14 04:49:09.863316 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.863279    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:49:09.941692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.941649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2d000910-baed-4c0c-90a7-607e2b0f8664-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:09.941973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:09.941715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d000910-baed-4c0c-90a7-607e2b0f8664-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.003581 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.003541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:10.042642 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.042560    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2d000910-baed-4c0c-90a7-607e2b0f8664-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.042642 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.042626    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d000910-baed-4c0c-90a7-607e2b0f8664-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.043216 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.043195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2d000910-baed-4c0c-90a7-607e2b0f8664-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.045242 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.045219    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d000910-baed-4c0c-90a7-607e2b0f8664-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-scrkt\" (UID: \"2d000910-baed-4c0c-90a7-607e2b0f8664\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.087340 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.087304    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"
Sep 14 04:49:10.106671 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.106632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:10.365703 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.365600    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt"]
Sep 14 04:49:10.369034 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:10.369007    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d000910_baed_4c0c_90a7_607e2b0f8664.slice/crio-3b41f80802ae1f2cfa320db5638e75d1791b7ae7163b8bf0c91c450806702bbe WatchSource:0}: Error finding container 3b41f80802ae1f2cfa320db5638e75d1791b7ae7163b8bf0c91c450806702bbe: Status 404 returned error can't find the container with id 3b41f80802ae1f2cfa320db5638e75d1791b7ae7163b8bf0c91c450806702bbe
Sep 14 04:49:10.702295 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.702262    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt" event={"ID":"2d000910-baed-4c0c-90a7-607e2b0f8664","Type":"ContainerStarted","Data":"3b41f80802ae1f2cfa320db5638e75d1791b7ae7163b8bf0c91c450806702bbe"}
Sep 14 04:49:10.971279 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:10.971174    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:11.083553 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.083515    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:11.215041 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.215001    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:49:11.215198 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.215072    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:49:11.215419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.215404    2571 scope.go:117] "RemoveContainer" containerID="c6909e957690e5218e274b6610e75d727d753f667b97bed87e7d1dbad2db284c"
Sep 14 04:49:11.215578 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:11.215561    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-dn8r4_openshift-console-operator(c016e1b5-cc4b-41f5-abe7-424d6a8023d7)\"" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podUID="c016e1b5-cc4b-41f5-abe7-424d6a8023d7"
Sep 14 04:49:11.706803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.706763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt" event={"ID":"2d000910-baed-4c0c-90a7-607e2b0f8664","Type":"ContainerStarted","Data":"f20470a55d7a4e1e830b3e92ede4407549878d4f74cd5e02eaeac4b98fb62221"}
Sep 14 04:49:11.726468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.726413    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-scrkt" podStartSLOduration=1.541869285 podStartE2EDuration="2.726396289s" totalImagesPullingTime="1.184527004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:11.725800304 +0000 UTC m=+146.087752758" watchObservedRunningTime="2026-09-14 04:49:11.726396289 +0000 UTC m=+146.088348731"
Sep 14 04:49:11.849078 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.848978    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:11.915560 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:11.915527    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:13.168486 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:13.168452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:15.146920 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:15.146880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:15.259626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:15.259590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:15.353115 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:15.353081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:15.447407 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:15.447325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:16.895045 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:16.895002    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:16.897609 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:16.897584    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5369a29b-707b-42ed-a29c-f34c71fce8f8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-55sbp\" (UID: \"5369a29b-707b-42ed-a29c-f34c71fce8f8\") " pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:17.172996 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:17.172902    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d8h7x\""
Sep 14 04:49:17.180357 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:17.180328    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-55sbp"
Sep 14 04:49:17.453490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:17.453457    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-55sbp"]
Sep 14 04:49:17.456562 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:17.456529    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5369a29b_707b_42ed_a29c_f34c71fce8f8.slice/crio-c036f056076c7f508f5575a9afea454b657922d263c2d9e0d618a4c8827aacab WatchSource:0}: Error finding container c036f056076c7f508f5575a9afea454b657922d263c2d9e0d618a4c8827aacab: Status 404 returned error can't find the container with id c036f056076c7f508f5575a9afea454b657922d263c2d9e0d618a4c8827aacab
Sep 14 04:49:17.725328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:17.725250    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55sbp" event={"ID":"5369a29b-707b-42ed-a29c-f34c71fce8f8","Type":"ContainerStarted","Data":"174af2c0ae8c2b5ae927574b2085717064b861bf9379703dea61523bc4baf00e"}
Sep 14 04:49:17.725328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:17.725305    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55sbp" event={"ID":"5369a29b-707b-42ed-a29c-f34c71fce8f8","Type":"ContainerStarted","Data":"c036f056076c7f508f5575a9afea454b657922d263c2d9e0d618a4c8827aacab"}
Sep 14 04:49:18.408708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.408682    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:18.480542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.480506    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"]
Sep 14 04:49:18.483648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.483630    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.486215 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.486192    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 04:49:18.487247 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.487227    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 04:49:18.487367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.487225    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 04:49:18.487367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.487337    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 04:49:18.487485 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.487452    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 04:49:18.487485 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.487472    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jfn7g\""
Sep 14 04:49:18.493638 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.493609    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"]
Sep 14 04:49:18.497372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.497328    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:18.610772 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.610673    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.610772 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.610712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.610772 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.610735    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.611028 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.610841    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgk5x\" (UniqueName: \"kubernetes.io/projected/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-kube-api-access-jgk5x\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.711230 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.711201    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgk5x\" (UniqueName: \"kubernetes.io/projected/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-kube-api-access-jgk5x\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.711309 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.711255    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.711309 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.711276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.711309 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.711298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.711404 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:18.711365    2571 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 14 04:49:18.711459 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:18.711446    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls podName:38993dd4-9b2f-4abd-b9de-9332f1b6ac5b nodeName:}" failed. No retries permitted until 2026-09-14 04:49:19.211425276 +0000 UTC m=+153.573377713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-vx2s4" (UID: "38993dd4-9b2f-4abd-b9de-9332f1b6ac5b") : secret "prometheus-operator-tls" not found
Sep 14 04:49:18.712056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.712036    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.713791 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.713774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.720565 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.720538    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgk5x\" (UniqueName: \"kubernetes.io/projected/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-kube-api-access-jgk5x\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:18.729886 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.729854    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55sbp" event={"ID":"5369a29b-707b-42ed-a29c-f34c71fce8f8","Type":"ContainerStarted","Data":"0e87830dbce5300bc68321705e2d61abb3637d48e65e252cceeb5b85d69cb06d"}
Sep 14 04:49:18.856834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:18.856797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:19.215744 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.215710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:19.218972 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.218908    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/38993dd4-9b2f-4abd-b9de-9332f1b6ac5b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vx2s4\" (UID: \"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:19.393218 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.393177    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"
Sep 14 04:49:19.674572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.674487    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4"]
Sep 14 04:49:19.817716 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:19.817679    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38993dd4_9b2f_4abd_b9de_9332f1b6ac5b.slice/crio-85999b30a0ea5b375769ee76f05227ecbef6a59469ebf921ff7598f05cbf037c WatchSource:0}: Error finding container 85999b30a0ea5b375769ee76f05227ecbef6a59469ebf921ff7598f05cbf037c: Status 404 returned error can't find the container with id 85999b30a0ea5b375769ee76f05227ecbef6a59469ebf921ff7598f05cbf037c
Sep 14 04:49:19.869467 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.869438    2571 patch_prober.go:28] interesting pod/image-registry-6894f7db9d-64t5p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 14 04:49:19.869592 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:19.869490    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 04:49:20.740150 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:20.740110    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-55sbp" event={"ID":"5369a29b-707b-42ed-a29c-f34c71fce8f8","Type":"ContainerStarted","Data":"027c6a046462f8031704babc9fbd40c394aa648e06b76e56c9382ce854676cf9"}
Sep 14 04:49:20.741900 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:20.741867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4" event={"ID":"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b","Type":"ContainerStarted","Data":"85999b30a0ea5b375769ee76f05227ecbef6a59469ebf921ff7598f05cbf037c"}
Sep 14 04:49:20.760221 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:20.760162    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-55sbp" podStartSLOduration=33.470453271 podStartE2EDuration="35.760145049s" totalImagesPullingTime="2.289691778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 04:48:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:20.759944741 +0000 UTC m=+155.121897176" watchObservedRunningTime="2026-09-14 04:49:20.760145049 +0000 UTC m=+155.122097492"
Sep 14 04:49:21.402083 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:21.401988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:21.747448 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:21.747407    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4" event={"ID":"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b","Type":"ContainerStarted","Data":"66aa1a3391c28f5f1abbed4f2ea25a98d5dfee7ab63ed316708ef35b357c79fb"}
Sep 14 04:49:21.747918 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:21.747495    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4" event={"ID":"38993dd4-9b2f-4abd-b9de-9332f1b6ac5b","Type":"ContainerStarted","Data":"476f412108326d82f38891b4e54910ec04e0eb9fd2efb5d361f2dec516f93d62"}
Sep 14 04:49:21.768271 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:21.768218    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vx2s4" podStartSLOduration=2.482492982 podStartE2EDuration="3.768201012s" totalImagesPullingTime="1.28570803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:21.767566599 +0000 UTC m=+156.129519053" watchObservedRunningTime="2026-09-14 04:49:21.768201012 +0000 UTC m=+156.130153458"
Sep 14 04:49:23.585221 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.585182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:23.798844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.798807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:23.904564 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.904478    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cslbk"]
Sep 14 04:49:23.907727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.907705    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.910427 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.910404    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 04:49:23.910552 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.910536    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 04:49:23.910602 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.910559    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vr67q\""
Sep 14 04:49:23.910656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.910538    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 04:49:23.954265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954223    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-textfile\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954271    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-accelerators-collector-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-sys\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954378    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-wtmp\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-metrics-client-ca\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954586 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954449    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5b4b\" (UniqueName: \"kubernetes.io/projected/f1554549-75e9-4d27-b4c5-c8b2d0e58201-kube-api-access-p5b4b\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954586 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-root\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.954586 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.954501    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-tls\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:23.998437 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:23.998401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.042862 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:24.042822    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-zw87b" podUID="f3bebbe5-347c-46df-b90e-542a9c8882db"
Sep 14 04:49:24.055067 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-wtmp\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055075    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055244 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055206    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-wtmp\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055244 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-metrics-client-ca\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055305 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055263    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5b4b\" (UniqueName: \"kubernetes.io/projected/f1554549-75e9-4d27-b4c5-c8b2d0e58201-kube-api-access-p5b4b\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055305 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-root\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055387 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055309    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-tls\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055387 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-textfile\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055387 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-accelerators-collector-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055404    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-sys\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055408    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-root\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055509    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1554549-75e9-4d27-b4c5-c8b2d0e58201-sys\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.055824 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055765    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-metrics-client-ca\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.056052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055846    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-textfile\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.056052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.055962    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-accelerators-collector-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.057627 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.057602    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.057722 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.057654    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1554549-75e9-4d27-b4c5-c8b2d0e58201-node-exporter-tls\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.068142 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.068118    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5b4b\" (UniqueName: \"kubernetes.io/projected/f1554549-75e9-4d27-b4c5-c8b2d0e58201-kube-api-access-p5b4b\") pod \"node-exporter-cslbk\" (UID: \"f1554549-75e9-4d27-b4c5-c8b2d0e58201\") " pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.081432 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.081396    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.184312 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.184198    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.216926 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.216886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cslbk"
Sep 14 04:49:24.229290 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:24.229264    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1554549_75e9_4d27_b4c5_c8b2d0e58201.slice/crio-13098ccf90de993642431c1dc6bf8cf49579e3cf0d15e969e33bf011dcb9a1c5 WatchSource:0}: Error finding container 13098ccf90de993642431c1dc6bf8cf49579e3cf0d15e969e33bf011dcb9a1c5: Status 404 returned error can't find the container with id 13098ccf90de993642431c1dc6bf8cf49579e3cf0d15e969e33bf011dcb9a1c5
Sep 14 04:49:24.251794 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.251761    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.363294 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.363264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.758365 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.758332    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:24.758763 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.758326    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cslbk" event={"ID":"f1554549-75e9-4d27-b4c5-c8b2d0e58201","Type":"ContainerStarted","Data":"13098ccf90de993642431c1dc6bf8cf49579e3cf0d15e969e33bf011dcb9a1c5"}
Sep 14 04:49:24.823334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.823299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:24.921470 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:24.921429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:25.212356 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:25.212305    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9nbn5" podUID="f90d5ac3-fe7e-4f0f-83bf-33aa002548bc"
Sep 14 04:49:25.585619 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:25.585579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:25.763404 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:25.763305    2571 generic.go:356] "Generic (PLEG): container finished" podID="f1554549-75e9-4d27-b4c5-c8b2d0e58201" containerID="719985ea27e99e2b35191c2a2859ea1b85e97ea19392b427e93cceb8407d4987" exitCode=0
Sep 14 04:49:25.763404 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:25.763391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cslbk" event={"ID":"f1554549-75e9-4d27-b4c5-c8b2d0e58201","Type":"ContainerDied","Data":"719985ea27e99e2b35191c2a2859ea1b85e97ea19392b427e93cceb8407d4987"}
Sep 14 04:49:26.189589 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.189501    2571 scope.go:117] "RemoveContainer" containerID="c6909e957690e5218e274b6610e75d727d753f667b97bed87e7d1dbad2db284c"
Sep 14 04:49:26.769389 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.769358    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:49:26.769782 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.769457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" event={"ID":"c016e1b5-cc4b-41f5-abe7-424d6a8023d7","Type":"ContainerStarted","Data":"06bd4b4233598d8c58b4aad00bc6df7b25015b0820e30e4874d2b2fa1e0447b8"}
Sep 14 04:49:26.769782 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.769764    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:49:26.772032 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.772002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cslbk" event={"ID":"f1554549-75e9-4d27-b4c5-c8b2d0e58201","Type":"ContainerStarted","Data":"383ff4629264f2fd5c0dc293394e129aae99563433d78584eb4be73139f1bdd7"}
Sep 14 04:49:26.772157 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.772061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cslbk" event={"ID":"f1554549-75e9-4d27-b4c5-c8b2d0e58201","Type":"ContainerStarted","Data":"ef40b140acb6ad71a30854e1802d5125c4a0fd0cdad0bfe8dabed94adc50e15c"}
Sep 14 04:49:26.775048 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.775027    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4"
Sep 14 04:49:26.796611 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.796561    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-dn8r4" podStartSLOduration=44.468315466 podStartE2EDuration="46.796542365s" totalImagesPullingTime="2.328226899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:48:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:26.796119793 +0000 UTC m=+161.158072237" watchObservedRunningTime="2026-09-14 04:49:26.796542365 +0000 UTC m=+161.158494807"
Sep 14 04:49:26.806863 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.806834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:26.853221 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.853156    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cslbk" podStartSLOduration=2.675060115 podStartE2EDuration="3.853130644s" totalImagesPullingTime="1.106339369s" totalInitContainerRuntime="71.73116ms" isStatefulPod=true podCreationTimestamp="2026-09-14 04:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:26.851064699 +0000 UTC m=+161.213017135" watchObservedRunningTime="2026-09-14 04:49:26.853130644 +0000 UTC m=+161.215083088"
Sep 14 04:49:26.887321 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.887287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:26.939554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.939520    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"]
Sep 14 04:49:26.943020 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.943001    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.945692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.945669    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 04:49:26.946029 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946003    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kbqvr\""
Sep 14 04:49:26.946113 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946045    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 04:49:26.946388 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946367    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 04:49:26.946482 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946422    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 04:49:26.946482 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946430    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 04:49:26.946482 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.946444    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-34k3vfo6b6lla\""
Sep 14 04:49:26.964693 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.964662    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"]
Sep 14 04:49:26.981785 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-grpc-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.981785 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981817    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981859    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.981966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a70088-ecb8-4a11-8c4d-32a79e686fb7-metrics-client-ca\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.982039    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.982097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.982078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q58h\" (UniqueName: \"kubernetes.io/projected/96a70088-ecb8-4a11-8c4d-32a79e686fb7-kube-api-access-9q58h\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:26.985430 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:26.985405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.062076 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.061996    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.083424 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-grpc-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083567 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083429    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083567 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083464    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083567 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083567 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083567 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a70088-ecb8-4a11-8c4d-32a79e686fb7-metrics-client-ca\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083815 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083679    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.083815 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.083710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q58h\" (UniqueName: \"kubernetes.io/projected/96a70088-ecb8-4a11-8c4d-32a79e686fb7-kube-api-access-9q58h\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.084561 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.084426    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96a70088-ecb8-4a11-8c4d-32a79e686fb7-metrics-client-ca\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.086520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.086488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.086626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.086528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.086789 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.086766    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.086826 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.086803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.087003 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.086979    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.087090 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.087075    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96a70088-ecb8-4a11-8c4d-32a79e686fb7-secret-grpc-tls\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.092441 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.092416    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q58h\" (UniqueName: \"kubernetes.io/projected/96a70088-ecb8-4a11-8c4d-32a79e686fb7-kube-api-access-9q58h\") pod \"thanos-querier-74b6cc799d-cqsm7\" (UID: \"96a70088-ecb8-4a11-8c4d-32a79e686fb7\") " pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.249488 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.249424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.252456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.252433    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:27.315948 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.315851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.458533 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.458494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.539019 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.538989    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"]
Sep 14 04:49:27.542612 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:27.542585    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a70088_ecb8_4a11_8c4d_32a79e686fb7.slice/crio-1294137c81a5dd87804ae5d17cb9c2255a002e2326595465dade66149e523d59 WatchSource:0}: Error finding container 1294137c81a5dd87804ae5d17cb9c2255a002e2326595465dade66149e523d59: Status 404 returned error can't find the container with id 1294137c81a5dd87804ae5d17cb9c2255a002e2326595465dade66149e523d59
Sep 14 04:49:27.629070 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.628968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.700889 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.700854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:27.777596 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:27.777559    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"1294137c81a5dd87804ae5d17cb9c2255a002e2326595465dade66149e523d59"}
Sep 14 04:49:28.024979 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.024920    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.098588 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.098552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.607670 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.607642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.626121 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.626078    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"]
Sep 14 04:49:28.631366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.631333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.635607 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.635580    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 14 04:49:28.635760 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.635641    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cn8rw\""
Sep 14 04:49:28.640529 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.640500    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"]
Sep 14 04:49:28.698026 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.697995    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.698207 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.698097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsndn\" (UniqueName: \"kubernetes.io/projected/83daf0c8-91ee-4345-8ef2-97628c929cfe-kube-api-access-xsndn\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.707648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.707616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.799203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.799164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsndn\" (UniqueName: \"kubernetes.io/projected/83daf0c8-91ee-4345-8ef2-97628c929cfe-kube-api-access-xsndn\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.799616 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.799225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.799616 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:28.799347    2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 14 04:49:28.799616 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:28.799411    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert podName:83daf0c8-91ee-4345-8ef2-97628c929cfe nodeName:}" failed. No retries permitted until 2026-09-14 04:49:29.299391773 +0000 UTC m=+163.661344194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-f2h9t" (UID: "83daf0c8-91ee-4345-8ef2-97628c929cfe") : secret "monitoring-plugin-cert" not found
Sep 14 04:49:28.811445 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.811410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsndn\" (UniqueName: \"kubernetes.io/projected/83daf0c8-91ee-4345-8ef2-97628c929cfe-kube-api-access-xsndn\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:28.841774 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.841737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.900169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.900078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:28.903005 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.902973    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3bebbe5-347c-46df-b90e-542a9c8882db-metrics-tls\") pod \"dns-default-zw87b\" (UID: \"f3bebbe5-347c-46df-b90e-542a9c8882db\") " pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:28.942470 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.942427    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:28.962626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.962588    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wwfwc\""
Sep 14 04:49:28.970036 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:28.970003    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:29.043648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.043608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:29.270527 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.270484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:29.303152 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.303048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:29.305759 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.305726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83daf0c8-91ee-4345-8ef2-97628c929cfe-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-f2h9t\" (UID: \"83daf0c8-91ee-4345-8ef2-97628c929cfe\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:29.372791 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.372759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:29.503798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.503768    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zw87b"]
Sep 14 04:49:29.506121 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:29.506093    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3bebbe5_347c_46df_b90e_542a9c8882db.slice/crio-ce0d140e3ed84d6fc82a1bdae13e3e35b96fa789f60180410fd1dbc7ef440d09 WatchSource:0}: Error finding container ce0d140e3ed84d6fc82a1bdae13e3e35b96fa789f60180410fd1dbc7ef440d09: Status 404 returned error can't find the container with id ce0d140e3ed84d6fc82a1bdae13e3e35b96fa789f60180410fd1dbc7ef440d09
Sep 14 04:49:29.547303 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.547263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:29.786608 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.786522    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zw87b" event={"ID":"f3bebbe5-347c-46df-b90e-542a9c8882db","Type":"ContainerStarted","Data":"ce0d140e3ed84d6fc82a1bdae13e3e35b96fa789f60180410fd1dbc7ef440d09"}
Sep 14 04:49:29.788874 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.788843    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"89f76389fdff434b6ae4ed2d25c418911e84ffeb902a005ea7f4f948e9e87232"}
Sep 14 04:49:29.789075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.788917    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"f9103eea383fe51b74eaf78a8d1b894713500876144d5f466cff6f66ed92c3e9"}
Sep 14 04:49:29.789075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.788966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"3168fbb6c6588bfda281e5bf41d1370e0344478e6695bd2cca5b9361ca02df53"}
Sep 14 04:49:29.818881 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.818846    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"]
Sep 14 04:49:29.822210 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:29.822175    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83daf0c8_91ee_4345_8ef2_97628c929cfe.slice/crio-a0db9ee0323db4d905cd9a7c89018f871dc9b24ff2a0acf6b38694b6b25fea64 WatchSource:0}: Error finding container a0db9ee0323db4d905cd9a7c89018f871dc9b24ff2a0acf6b38694b6b25fea64: Status 404 returned error can't find the container with id a0db9ee0323db4d905cd9a7c89018f871dc9b24ff2a0acf6b38694b6b25fea64
Sep 14 04:49:29.867886 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:29.867847    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:49:30.132495 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:30.132360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:30.798650 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:30.798565    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"8c793ed046bfedf0388f562ca120c2775def4734c5916a08486be98a226b0d2a"}
Sep 14 04:49:30.798650 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:30.798644    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"0ff90069614327f88bd92b1029f120e71d2187f76e645df9f87d7e0d35ba23cb"}
Sep 14 04:49:30.800248 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:30.800204    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t" event={"ID":"83daf0c8-91ee-4345-8ef2-97628c929cfe","Type":"ContainerStarted","Data":"a0db9ee0323db4d905cd9a7c89018f871dc9b24ff2a0acf6b38694b6b25fea64"}
Sep 14 04:49:31.807372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.807332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" event={"ID":"96a70088-ecb8-4a11-8c4d-32a79e686fb7","Type":"ContainerStarted","Data":"0a45c3e7104576c8d22bbc5b86184b1c3067a646887de2756db502234f941b90"}
Sep 14 04:49:31.807810 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.807470    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:31.808990 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.808965    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zw87b" event={"ID":"f3bebbe5-347c-46df-b90e-542a9c8882db","Type":"ContainerStarted","Data":"c305b5cb804dbfdd0fd0d03c91c8e8e3da0eff5ae66740034effb67aca92db3f"}
Sep 14 04:49:31.810467 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.810445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t" event={"ID":"83daf0c8-91ee-4345-8ef2-97628c929cfe","Type":"ContainerStarted","Data":"d7ce66fdf1abb50d0493aa9e60bdafc3aa8301e35c99c0a2c7df3465a93398fc"}
Sep 14 04:49:31.810615 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.810596    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:31.815224 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.815201    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t"
Sep 14 04:49:31.832244 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.832192    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7" podStartSLOduration=3.020592511 podStartE2EDuration="5.832157099s" totalImagesPullingTime="2.811564588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:31.831036681 +0000 UTC m=+166.192989123" watchObservedRunningTime="2026-09-14 04:49:31.832157099 +0000 UTC m=+166.194109523"
Sep 14 04:49:31.846406 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:31.846335    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-f2h9t" podStartSLOduration=2.008237295 podStartE2EDuration="3.84631412s" totalImagesPullingTime="1.838076825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:31.845609398 +0000 UTC m=+166.207561842" watchObservedRunningTime="2026-09-14 04:49:31.84631412 +0000 UTC m=+166.208266598"
Sep 14 04:49:32.161483 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:32.161453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:32.256350 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:32.256318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:32.816660 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:32.816626    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zw87b" event={"ID":"f3bebbe5-347c-46df-b90e-542a9c8882db","Type":"ContainerStarted","Data":"1f6832acfba157305c17e3d64cb0b6575f7d4d70985943126a30910880772a97"}
Sep 14 04:49:32.817110 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:32.816725    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:32.835496 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:32.835443    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zw87b" podStartSLOduration=130.725470506 podStartE2EDuration="2m12.835423687s" totalImagesPullingTime="2.109953181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:32.834502945 +0000 UTC m=+167.196455391" watchObservedRunningTime="2026-09-14 04:49:32.835423687 +0000 UTC m=+167.197376129"
Sep 14 04:49:33.305897 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:33.305865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:33.369516 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:33.369487    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:34.884413 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:34.884366    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerName="registry" containerID="cri-o://5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354" gracePeriod=30
Sep 14 04:49:35.146060 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.145987    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:49:35.256182 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256132    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256230    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256271    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256316    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256362    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256551 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256386    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256551 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256416    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7qsz\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256551 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256460    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted\") pod \"f0add8d8-f38a-4337-842c-d1824f1e521c\" (UID: \"f0add8d8-f38a-4337-842c-d1824f1e521c\") "
Sep 14 04:49:35.256730 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256551    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:49:35.256730 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256716    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-certificates\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.256835 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.256726    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:49:35.259400 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.259355    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:49:35.259400 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.259372    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz" (OuterVolumeSpecName: "kube-api-access-b7qsz") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "kube-api-access-b7qsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:49:35.259400 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.259373    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:49:35.259665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.259562    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:49:35.259996 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.259963    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:49:35.267314 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.267275    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f0add8d8-f38a-4337-842c-d1824f1e521c" (UID: "f0add8d8-f38a-4337-842c-d1824f1e521c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:49:35.357689 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357651    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-bound-sa-token\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357689 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357681    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-installation-pull-secrets\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357689 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357691    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-registry-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357701    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7qsz\" (UniqueName: \"kubernetes.io/projected/f0add8d8-f38a-4337-842c-d1824f1e521c-kube-api-access-b7qsz\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357709    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f0add8d8-f38a-4337-842c-d1824f1e521c-ca-trust-extracted\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357718    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f0add8d8-f38a-4337-842c-d1824f1e521c-image-registry-private-configuration\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.357915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.357750    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f0add8d8-f38a-4337-842c-d1824f1e521c-trusted-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:49:35.830015 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.829975    2571 generic.go:356] "Generic (PLEG): container finished" podID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerID="5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354" exitCode=0
Sep 14 04:49:35.830229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.830022    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p"
Sep 14 04:49:35.830229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.830061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" event={"ID":"f0add8d8-f38a-4337-842c-d1824f1e521c","Type":"ContainerDied","Data":"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"}
Sep 14 04:49:35.830229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.830117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6894f7db9d-64t5p" event={"ID":"f0add8d8-f38a-4337-842c-d1824f1e521c","Type":"ContainerDied","Data":"e6290e254f26559caf4efca80f1f63426e1f269c597d4b89d5a56ec43473973d"}
Sep 14 04:49:35.830229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.830142    2571 scope.go:117] "RemoveContainer" containerID="5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"
Sep 14 04:49:35.838817 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.838797    2571 scope.go:117] "RemoveContainer" containerID="5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"
Sep 14 04:49:35.839152 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:49:35.839129    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354\": container with ID starting with 5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354 not found: ID does not exist" containerID="5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"
Sep 14 04:49:35.839243 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.839160    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354"} err="failed to get container status \"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354\": rpc error: code = NotFound desc = could not find container \"5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354\": container with ID starting with 5ec31327cd3e6330ee19ae910c55d313c41e77cc5bfcca5de1d94041d1c09354 not found: ID does not exist"
Sep 14 04:49:35.852694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.852664    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:49:35.857504 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:35.857480    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6894f7db9d-64t5p"]
Sep 14 04:49:36.192661 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.192582    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" path="/var/lib/kubelet/pods/f0add8d8-f38a-4337-842c-d1824f1e521c/volumes"
Sep 14 04:49:36.206415 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.206377    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:49:36.206661 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.206650    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerName="registry"
Sep 14 04:49:36.206712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.206663    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerName="registry"
Sep 14 04:49:36.206712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.206708    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0add8d8-f38a-4337-842c-d1824f1e521c" containerName="registry"
Sep 14 04:49:36.210803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.210783    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.213648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.213621    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 04:49:36.213790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.213699    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 04:49:36.214571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214555    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 04:49:36.214658 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214604    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 04:49:36.214658 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214561    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cs9m4\""
Sep 14 04:49:36.214995 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214973    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 04:49:36.215131 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214981    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 04:49:36.215131 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.214982    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 04:49:36.220799 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.220773    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:49:36.266506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266462    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.266696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266519    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wx6\" (UniqueName: \"kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.266696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.266696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266635    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.266696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.266696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.266697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367525 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367525 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wx6\" (UniqueName: \"kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.367976 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.367799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.368420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.368333    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.368420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.368382    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.368420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.368431    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.370281 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.370252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.370388 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.370252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.376906 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.376885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wx6\" (UniqueName: \"kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6\") pod \"console-654d88987-cnvv5\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") " pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.521264 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.521218    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:36.816266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.816211    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:49:36.818240 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:36.818215    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2983737b_74c3_4189_b6b6_49a78537549b.slice/crio-1fd89e00b0d91af7ba87395c905205373a6a3f11ac2e628ddbba97f82dcf878e WatchSource:0}: Error finding container 1fd89e00b0d91af7ba87395c905205373a6a3f11ac2e628ddbba97f82dcf878e: Status 404 returned error can't find the container with id 1fd89e00b0d91af7ba87395c905205373a6a3f11ac2e628ddbba97f82dcf878e
Sep 14 04:49:36.835398 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:36.835364    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654d88987-cnvv5" event={"ID":"2983737b-74c3-4189-b6b6-49a78537549b","Type":"ContainerStarted","Data":"1fd89e00b0d91af7ba87395c905205373a6a3f11ac2e628ddbba97f82dcf878e"}
Sep 14 04:49:37.188145 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:37.188062    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:49:37.823957 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:37.823892    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-74b6cc799d-cqsm7"
Sep 14 04:49:39.850027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:39.849917    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654d88987-cnvv5" event={"ID":"2983737b-74c3-4189-b6b6-49a78537549b","Type":"ContainerStarted","Data":"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"}
Sep 14 04:49:39.872116 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:39.872062    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654d88987-cnvv5" podStartSLOduration=1.159163001 podStartE2EDuration="3.872046598s" totalImagesPullingTime="2.712883597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:39.870077269 +0000 UTC m=+174.232029711" watchObservedRunningTime="2026-09-14 04:49:39.872046598 +0000 UTC m=+174.233999041"
Sep 14 04:49:40.471097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:40.471067    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:40.536371 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:40.536332    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:42.439816 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:42.439777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:42.551443 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:42.551405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:42.822700 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:42.822672    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zw87b"
Sep 14 04:49:43.870658 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:43.870600    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:43.938172 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:43.938132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:44.071791 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.071751    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:49:44.075217 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.075197    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.084379 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.084352    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 04:49:44.090627 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.090598    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:49:44.143038 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.142919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143038 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.142982    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143231 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.143082    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143231 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.143125    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143231 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.143164    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143231 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.143210    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.143356 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.143235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjwnc\" (UniqueName: \"kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.200016 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.199980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:44.244305 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244272    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244417 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244417 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244417 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjwnc\" (UniqueName: \"kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.244572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.244469    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.245193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.245166    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.245319 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.245212    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.245319 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.245270    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.245421 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.245362    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.247007 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.246981    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.247007 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.246995    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.256910 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.256882    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjwnc\" (UniqueName: \"kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc\") pod \"console-9ff9f8fdb-hzqz8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") " pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.269440 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.269410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:44.385224 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.385182    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:44.662380 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.662287    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:49:44.665112 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:49:44.665080    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c14029f_3a70_4ff3_9dae_8b459b6a5ac8.slice/crio-da3329b9f95c54657824af074d277266793028a9933cc3ba95359abe98926b3f WatchSource:0}: Error finding container da3329b9f95c54657824af074d277266793028a9933cc3ba95359abe98926b3f: Status 404 returned error can't find the container with id da3329b9f95c54657824af074d277266793028a9933cc3ba95359abe98926b3f
Sep 14 04:49:44.869854 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.869816    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9ff9f8fdb-hzqz8" event={"ID":"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8","Type":"ContainerStarted","Data":"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"}
Sep 14 04:49:44.870256 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.869877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9ff9f8fdb-hzqz8" event={"ID":"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8","Type":"ContainerStarted","Data":"da3329b9f95c54657824af074d277266793028a9933cc3ba95359abe98926b3f"}
Sep 14 04:49:44.891286 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:44.891231    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9ff9f8fdb-hzqz8" podStartSLOduration=0.891214243 podStartE2EDuration="891.214243ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:49:44.890182924 +0000 UTC m=+179.252135369" watchObservedRunningTime="2026-09-14 04:49:44.891214243 +0000 UTC m=+179.253166686"
Sep 14 04:49:45.201372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:45.201336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:45.275903 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:45.275866    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:45.488832 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:45.488797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:46.521871 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:46.521835    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:46.522337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:46.521913    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:46.526809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:46.526789    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:46.881650 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:46.881573    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:49:48.072948 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:48.072897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:48.171707 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:48.171679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:50.315593 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:50.315503    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:50.386380 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:50.386347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:50.892642 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:50.892601    2571 generic.go:356] "Generic (PLEG): container finished" podID="27a1027d-4dc2-4f57-a251-57eb0b9e95b3" containerID="c6ea17ab59d9eb3e6e303b102fa3a7cef4103d105c434209cc0aaff72710f8ca" exitCode=0
Sep 14 04:49:50.892820 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:50.892680    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh" event={"ID":"27a1027d-4dc2-4f57-a251-57eb0b9e95b3","Type":"ContainerDied","Data":"c6ea17ab59d9eb3e6e303b102fa3a7cef4103d105c434209cc0aaff72710f8ca"}
Sep 14 04:49:50.893093 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:50.893076    2571 scope.go:117] "RemoveContainer" containerID="c6ea17ab59d9eb3e6e303b102fa3a7cef4103d105c434209cc0aaff72710f8ca"
Sep 14 04:49:51.505924 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:51.505890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:51.639217 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:51.639173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:51.740104 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:51.740069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:51.836873 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:51.836781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:51.898751 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:51.898719    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-fsrgh" event={"ID":"27a1027d-4dc2-4f57-a251-57eb0b9e95b3","Type":"ContainerStarted","Data":"6de558e147bc35e97fd027bb817b6436185c24d5ab0ed1514c3eefd35a63f734"}
Sep 14 04:49:54.212894 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.212857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:54.386103 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.386069    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:54.386278 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.386133    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:54.390981 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.390926    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:54.914595 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.914566    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:49:54.969259 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:54.969224    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:49:55.108669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:55.108632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:55.209494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:55.209468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:55.278301 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:55.278243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:56.308816 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:56.308768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:56.410646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:56.410612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:49:56.512866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:49:56.512828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:10.974638 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:10.974606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:11.076859 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:11.076827    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:19.993947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:19.993870    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-654d88987-cnvv5" podUID="2983737b-74c3-4189-b6b6-49a78537549b" containerName="console" containerID="cri-o://39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d" gracePeriod=15
Sep 14 04:50:20.240667 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.240635    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:50:20.330344 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330251    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5wx6\" (UniqueName: \"kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330344 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330295    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330344 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330329    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330344 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330348    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330363    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330393    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config\") pod \"2983737b-74c3-4189-b6b6-49a78537549b\" (UID: \"2983737b-74c3-4189-b6b6-49a78537549b\") "
Sep 14 04:50:20.330830 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330799    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:50:20.330878 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330798    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca" (OuterVolumeSpecName: "service-ca") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:50:20.330916 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.330886    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config" (OuterVolumeSpecName: "console-config") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:50:20.332846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.332811    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:50:20.332846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.332834    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6" (OuterVolumeSpecName: "kube-api-access-b5wx6") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "kube-api-access-b5wx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:50:20.332846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.332833    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2983737b-74c3-4189-b6b6-49a78537549b" (UID: "2983737b-74c3-4189-b6b6-49a78537549b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:50:20.431809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431755    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5wx6\" (UniqueName: \"kubernetes.io/projected/2983737b-74c3-4189-b6b6-49a78537549b-kube-api-access-b5wx6\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:20.431809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431799    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:20.431809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431810    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2983737b-74c3-4189-b6b6-49a78537549b-console-oauth-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:20.431809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431820    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:20.431809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431829    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-oauth-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:20.432135 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:20.431837    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2983737b-74c3-4189-b6b6-49a78537549b-console-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:50:21.007694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.007655    2571 generic.go:356] "Generic (PLEG): container finished" podID="2983737b-74c3-4189-b6b6-49a78537549b" containerID="39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d" exitCode=0
Sep 14 04:50:21.007694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.007697    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654d88987-cnvv5"
Sep 14 04:50:21.008250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.007712    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654d88987-cnvv5" event={"ID":"2983737b-74c3-4189-b6b6-49a78537549b","Type":"ContainerDied","Data":"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"}
Sep 14 04:50:21.008250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.007784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654d88987-cnvv5" event={"ID":"2983737b-74c3-4189-b6b6-49a78537549b","Type":"ContainerDied","Data":"1fd89e00b0d91af7ba87395c905205373a6a3f11ac2e628ddbba97f82dcf878e"}
Sep 14 04:50:21.008250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.007806    2571 scope.go:117] "RemoveContainer" containerID="39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"
Sep 14 04:50:21.017126 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.017107    2571 scope.go:117] "RemoveContainer" containerID="39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"
Sep 14 04:50:21.017471 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:50:21.017449    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d\": container with ID starting with 39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d not found: ID does not exist" containerID="39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"
Sep 14 04:50:21.017527 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.017481    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d"} err="failed to get container status \"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d\": rpc error: code = NotFound desc = could not find container \"39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d\": container with ID starting with 39cf0aa8b84f366b5f7c7c6eed9962f6576e1d1ebdd9e03b5b404034cbf2299d not found: ID does not exist"
Sep 14 04:50:21.030581 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.030549    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:50:21.035598 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:21.035571    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654d88987-cnvv5"]
Sep 14 04:50:22.192680 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:22.192642    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2983737b-74c3-4189-b6b6-49a78537549b" path="/var/lib/kubelet/pods/2983737b-74c3-4189-b6b6-49a78537549b/volumes"
Sep 14 04:50:43.165352 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:43.165308    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:43.266833 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:43.266802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:43.723469 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:43.723435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:43.791646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:43.791570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:48.348168 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.348129    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"]
Sep 14 04:50:48.348548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.348406    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2983737b-74c3-4189-b6b6-49a78537549b" containerName="console"
Sep 14 04:50:48.348548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.348417    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2983737b-74c3-4189-b6b6-49a78537549b" containerName="console"
Sep 14 04:50:48.348548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.348468    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2983737b-74c3-4189-b6b6-49a78537549b" containerName="console"
Sep 14 04:50:48.352733 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.352711    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.356417 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.356379    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 14 04:50:48.356614 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.356503    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 14 04:50:48.356695 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.356620    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5w2jg\""
Sep 14 04:50:48.356918 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.356893    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 14 04:50:48.357233 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.357218    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 14 04:50:48.357692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.357677    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 14 04:50:48.362102 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.362080    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 14 04:50:48.367661 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.367634    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"]
Sep 14 04:50:48.466200 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-serving-certs-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466200 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466200 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466222    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-federate-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466315    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-metrics-client-ca\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8dtx\" (UniqueName: \"kubernetes.io/projected/ee281523-5d85-4599-b09a-ac9a6dea4498-kube-api-access-n8dtx\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.466520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.466396    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567644 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567602    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567692    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-serving-certs-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567712    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-federate-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.567843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567832    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-metrics-client-ca\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.568188 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.567863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8dtx\" (UniqueName: \"kubernetes.io/projected/ee281523-5d85-4599-b09a-ac9a6dea4498-kube-api-access-n8dtx\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.568694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.568663    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-metrics-client-ca\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.568945 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.568902    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-serving-certs-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.569062 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.568946    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.571229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.570851    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-federate-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.571229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.571070    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.571229 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.571147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-telemeter-client-tls\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.571449 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.571323    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ee281523-5d85-4599-b09a-ac9a6dea4498-secret-telemeter-client\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.580008 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.579982    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8dtx\" (UniqueName: \"kubernetes.io/projected/ee281523-5d85-4599-b09a-ac9a6dea4498-kube-api-access-n8dtx\") pod \"telemeter-client-7869cbc9fd-4tdhk\" (UID: \"ee281523-5d85-4599-b09a-ac9a6dea4498\") " pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.663194 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.663107    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"
Sep 14 04:50:48.963838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:48.963424    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk"]
Sep 14 04:50:49.112957 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:49.112905    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk" event={"ID":"ee281523-5d85-4599-b09a-ac9a6dea4498","Type":"ContainerStarted","Data":"814c10916c47fb2d51377f7c7dd7a0e024c331cd05fbd670728f5c7f650d07de"}
Sep 14 04:50:51.121906 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:51.121810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk" event={"ID":"ee281523-5d85-4599-b09a-ac9a6dea4498","Type":"ContainerStarted","Data":"0fc31bc594c9c6c46074cc88d208375c140904960e9c54851f08d6b55f39a69a"}
Sep 14 04:50:52.131327 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:52.131295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk" event={"ID":"ee281523-5d85-4599-b09a-ac9a6dea4498","Type":"ContainerStarted","Data":"3ccdcf239da9fd3cc37bd06dc90a8542422efd0eaab4cb5a743eafef782085dd"}
Sep 14 04:50:52.556670 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:52.556633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:53.136540 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.136507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk" event={"ID":"ee281523-5d85-4599-b09a-ac9a6dea4498","Type":"ContainerStarted","Data":"fbf706279c82a6ff8eb7d40bb257ec7724f4d4dae1572adc75ebfc6d0bcaac64"}
Sep 14 04:50:53.168027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.167973    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7869cbc9fd-4tdhk" podStartSLOduration=2.2236177010000002 podStartE2EDuration="5.167952882s" totalImagesPullingTime="2.944335181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:50:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:50:53.166869 +0000 UTC m=+247.528821454" watchObservedRunningTime="2026-09-14 04:50:53.167952882 +0000 UTC m=+247.529905322"
Sep 14 04:50:53.273502 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.273463    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:50:53.276521 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.276498    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.289694 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.289667    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:50:53.331889 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.331851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:53.412406 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412312    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412563 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412440    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412563 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxcs9\" (UniqueName: \"kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412563 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412521    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.412692 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.412677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.434847 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.434812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxcs9\" (UniqueName: \"kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513340    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513370    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.513406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.514235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.514844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.514812    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.515084 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.515044    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.515547 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.515526    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.516921 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.516896    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.517038 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.517028    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.523895 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.523871    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxcs9\" (UniqueName: \"kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9\") pod \"console-78fb7cf45d-mnzbp\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") " pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.536790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.536741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:53.586871 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.586825    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:50:53.884046 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:53.884003    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:50:53.886947 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:50:53.886898    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96f9765_f305_4a02_85e7_38c15308d2c5.slice/crio-66eab0ff294c8e0f6804d5fcd68e01763e083da976a234fca668fee8e9afb61f WatchSource:0}: Error finding container 66eab0ff294c8e0f6804d5fcd68e01763e083da976a234fca668fee8e9afb61f: Status 404 returned error can't find the container with id 66eab0ff294c8e0f6804d5fcd68e01763e083da976a234fca668fee8e9afb61f
Sep 14 04:50:54.068896 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:54.068861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:54.136843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:54.136750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:54.145708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:54.145670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fb7cf45d-mnzbp" event={"ID":"b96f9765-f305-4a02-85e7-38c15308d2c5","Type":"ContainerStarted","Data":"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"}
Sep 14 04:50:54.145870 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:54.145751    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fb7cf45d-mnzbp" event={"ID":"b96f9765-f305-4a02-85e7-38c15308d2c5","Type":"ContainerStarted","Data":"66eab0ff294c8e0f6804d5fcd68e01763e083da976a234fca668fee8e9afb61f"}
Sep 14 04:50:54.168530 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:54.168479    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78fb7cf45d-mnzbp" podStartSLOduration=1.168464322 podStartE2EDuration="1.168464322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:50:54.1675175 +0000 UTC m=+248.529469942" watchObservedRunningTime="2026-09-14 04:50:54.168464322 +0000 UTC m=+248.530416764"
Sep 14 04:50:55.468815 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:55.468775    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:55.537906 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:55.537838    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:50:58.050887 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:58.050849    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:50:58.053396 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:58.053375    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f90d5ac3-fe7e-4f0f-83bf-33aa002548bc-metrics-certs\") pod \"network-metrics-daemon-9nbn5\" (UID: \"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc\") " pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:50:58.191415 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:58.191375    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kgkvn\""
Sep 14 04:50:58.199808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:58.199781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9nbn5"
Sep 14 04:50:58.471138 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:58.471099    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9nbn5"]
Sep 14 04:50:58.478639 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:50:58.478608    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf90d5ac3_fe7e_4f0f_83bf_33aa002548bc.slice/crio-2b703492c3bd346982a04cecdaa938497b80c1e1f10d7925115079ef6a5db389 WatchSource:0}: Error finding container 2b703492c3bd346982a04cecdaa938497b80c1e1f10d7925115079ef6a5db389: Status 404 returned error can't find the container with id 2b703492c3bd346982a04cecdaa938497b80c1e1f10d7925115079ef6a5db389
Sep 14 04:50:59.168213 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:50:59.168160    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbn5" event={"ID":"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc","Type":"ContainerStarted","Data":"2b703492c3bd346982a04cecdaa938497b80c1e1f10d7925115079ef6a5db389"}
Sep 14 04:51:00.173984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:00.173948    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbn5" event={"ID":"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc","Type":"ContainerStarted","Data":"df1920a4b43629d14da355cde5fb5217e6d37e6a4f2629fae6e177cf96109fc2"}
Sep 14 04:51:00.174372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:00.174001    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9nbn5" event={"ID":"f90d5ac3-fe7e-4f0f-83bf-33aa002548bc","Type":"ContainerStarted","Data":"e4ee6fe1a914f37a62548eb4abd30426c2c5da686c9cba5e338de8f5555bbbe1"}
Sep 14 04:51:00.197724 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:00.197668    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9nbn5" podStartSLOduration=253.386025687 podStartE2EDuration="4m14.197649485s" totalImagesPullingTime="811.623798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:51:00.195309189 +0000 UTC m=+254.557261631" watchObservedRunningTime="2026-09-14 04:51:00.197649485 +0000 UTC m=+254.559601927"
Sep 14 04:51:00.307142 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:00.307105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:00.383498 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:00.383455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:01.642879 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:01.642844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:03.587967 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:03.587915    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:51:03.588372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:03.587988    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:51:03.592978 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:03.592947    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:51:04.198637 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:04.198602    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:51:04.248306 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:04.248272    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:51:04.364236 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:04.364201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:04.504020 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:04.503990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:04.574821 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:04.574779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:29.269200 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.269126    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9ff9f8fdb-hzqz8" podUID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" containerName="console" containerID="cri-o://8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc" gracePeriod=15
Sep 14 04:51:29.510594 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.510568    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:51:29.604997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.604872    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.604997 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.604922    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605024    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjwnc\" (UniqueName: \"kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605142    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605194    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605449 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605283    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert\") pod \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\" (UID: \"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8\") "
Sep 14 04:51:29.605449 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605428    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:51:29.605543 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605449    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config" (OuterVolumeSpecName: "console-config") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:51:29.605595 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605582    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.605648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605589    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca" (OuterVolumeSpecName: "service-ca") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:51:29.605648 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605602    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-trusted-ca-bundle\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.605918 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.605706    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:51:29.607326 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.607298    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc" (OuterVolumeSpecName: "kube-api-access-sjwnc") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "kube-api-access-sjwnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:51:29.607431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.607392    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:51:29.607484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.607464    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" (UID: "8c14029f-3a70-4ff3-9dae-8b459b6a5ac8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:51:29.706712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.706670    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.706712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.706704    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjwnc\" (UniqueName: \"kubernetes.io/projected/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-kube-api-access-sjwnc\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.706712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.706714    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.706712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.706723    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-console-oauth-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:29.707014 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:29.706731    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8-oauth-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:51:30.291374 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.291334    2571 generic.go:356] "Generic (PLEG): container finished" podID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" containerID="8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc" exitCode=0
Sep 14 04:51:30.291856 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.291382    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9ff9f8fdb-hzqz8"
Sep 14 04:51:30.291856 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.291416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9ff9f8fdb-hzqz8" event={"ID":"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8","Type":"ContainerDied","Data":"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"}
Sep 14 04:51:30.291856 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.291481    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9ff9f8fdb-hzqz8" event={"ID":"8c14029f-3a70-4ff3-9dae-8b459b6a5ac8","Type":"ContainerDied","Data":"da3329b9f95c54657824af074d277266793028a9933cc3ba95359abe98926b3f"}
Sep 14 04:51:30.291856 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.291504    2571 scope.go:117] "RemoveContainer" containerID="8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"
Sep 14 04:51:30.300035 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.300005    2571 scope.go:117] "RemoveContainer" containerID="8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"
Sep 14 04:51:30.300366 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:51:30.300346    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc\": container with ID starting with 8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc not found: ID does not exist" containerID="8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"
Sep 14 04:51:30.300414 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.300378    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc"} err="failed to get container status \"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc\": rpc error: code = NotFound desc = could not find container \"8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc\": container with ID starting with 8ae62ff1bddf9f214a7f56d52fb7312737ea2b68830c24f0da8eb86b758f9ecc not found: ID does not exist"
Sep 14 04:51:30.312812 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.312779    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:51:30.317795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.317757    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9ff9f8fdb-hzqz8"]
Sep 14 04:51:30.429727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:30.429692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:32.192435 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:32.192399    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" path="/var/lib/kubelet/pods/8c14029f-3a70-4ff3-9dae-8b459b6a5ac8/volumes"
Sep 14 04:51:46.059780 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:46.059745    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:51:46.065362 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:46.065334    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:51:46.066978 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:46.066949    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:51:46.071211 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:46.071189    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 04:51:46.072505 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:46.072482    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:51:58.462008 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:58.461963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:51:58.531044 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:51:58.530999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:10.569147 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:10.569103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:10.666207 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:10.666177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:23.727855 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.727817    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:52:23.728328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.728209    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" containerName="console"
Sep 14 04:52:23.728328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.728227    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" containerName="console"
Sep 14 04:52:23.728328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.728314    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c14029f-3a70-4ff3-9dae-8b459b6a5ac8" containerName="console"
Sep 14 04:52:23.731330 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.731306    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.743087 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.743055    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:52:23.777064 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.777026    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:23.827248 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827262    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827289    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827328    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcrxw\" (UniqueName: \"kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827540 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.827540 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.827444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928544    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcrxw\" (UniqueName: \"kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928860 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928844    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.928920 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928906    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.929006 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.928975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.929272 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.929254    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.929419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.929386    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.929846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.929821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.929959 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.929921    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.931518 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.931491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.931631 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.931535    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:23.938499 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:23.938473    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcrxw\" (UniqueName: \"kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw\") pod \"console-88878b64b-6hnwb\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") " pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:24.042139 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:24.042102    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:24.318208 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:24.318126    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:52:24.321414 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:52:24.321379    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6eecf13_a29f_4f5d_bd2d_491d0b7d83c6.slice/crio-2a0b7161b6a40157e3138a680a59dd959cb768c283feeb3d11fb1ebdf8f63676 WatchSource:0}: Error finding container 2a0b7161b6a40157e3138a680a59dd959cb768c283feeb3d11fb1ebdf8f63676: Status 404 returned error can't find the container with id 2a0b7161b6a40157e3138a680a59dd959cb768c283feeb3d11fb1ebdf8f63676
Sep 14 04:52:24.505007 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:24.504972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88878b64b-6hnwb" event={"ID":"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6","Type":"ContainerStarted","Data":"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"}
Sep 14 04:52:24.505179 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:24.505018    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88878b64b-6hnwb" event={"ID":"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6","Type":"ContainerStarted","Data":"2a0b7161b6a40157e3138a680a59dd959cb768c283feeb3d11fb1ebdf8f63676"}
Sep 14 04:52:24.532770 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:24.532712    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-88878b64b-6hnwb" podStartSLOduration=1.5326904369999999 podStartE2EDuration="1.532690437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:52:24.532320225 +0000 UTC m=+338.894272668" watchObservedRunningTime="2026-09-14 04:52:24.532690437 +0000 UTC m=+338.894642882"
Sep 14 04:52:25.969176 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:25.969139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:26.069632 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:26.069604    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:26.170586 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:26.170555    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:28.656667 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:28.656616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:28.756136 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:28.756101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:34.042793 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.042750    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:34.042793 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.042812    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:34.047832 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.047808    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:34.546831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.546800    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:52:34.601616 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.601573    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:52:34.719390 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.719355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:34.781572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.781537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:34.846879 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:34.846801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:44.528606 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.528515    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:44.568735 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.568696    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"]
Sep 14 04:52:44.572225 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.572207    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.575082 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.575056    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 04:52:44.575808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.575791    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 04:52:44.576292 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.576279    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 04:52:44.577260 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.577234    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 04:52:44.577382 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.577267    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 04:52:44.577382 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.577269    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 04:52:44.577382 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.577277    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 04:52:44.584572 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.584543    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"]
Sep 14 04:52:44.601128 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.601099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:44.679213 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.679173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:52:44.699503 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699465    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/490fd034-c3ef-4dbc-903d-f63015f1a851-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.699503 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699504    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.699726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zmj\" (UniqueName: \"kubernetes.io/projected/490fd034-c3ef-4dbc-903d-f63015f1a851-kube-api-access-s5zmj\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.699726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699631    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.699726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699661    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.699726 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.699683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/490fd034-c3ef-4dbc-903d-f63015f1a851-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zmj\" (UniqueName: \"kubernetes.io/projected/490fd034-c3ef-4dbc-903d-f63015f1a851-kube-api-access-s5zmj\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800361    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.800659 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.800378    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.801054 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.801023    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/490fd034-c3ef-4dbc-903d-f63015f1a851-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.803085 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.803061    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.803183 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.803063    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-ca\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.803183 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.803153    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.803285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.803268    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/490fd034-c3ef-4dbc-903d-f63015f1a851-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.810022 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.809991    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zmj\" (UniqueName: \"kubernetes.io/projected/490fd034-c3ef-4dbc-903d-f63015f1a851-kube-api-access-s5zmj\") pod \"cluster-proxy-proxy-agent-bbb7db775-xx6ts\" (UID: \"490fd034-c3ef-4dbc-903d-f63015f1a851\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:44.898056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:44.897999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"
Sep 14 04:52:45.174122 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:45.174091    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts"]
Sep 14 04:52:45.176828 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:52:45.176798    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod490fd034_c3ef_4dbc_903d_f63015f1a851.slice/crio-4acb8eacddb87382680a16e85874618ffc8a754499672d2e756e4f77c3eda5db WatchSource:0}: Error finding container 4acb8eacddb87382680a16e85874618ffc8a754499672d2e756e4f77c3eda5db: Status 404 returned error can't find the container with id 4acb8eacddb87382680a16e85874618ffc8a754499672d2e756e4f77c3eda5db
Sep 14 04:52:45.178638 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:45.178619    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 04:52:45.591543 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:45.591509    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts" event={"ID":"490fd034-c3ef-4dbc-903d-f63015f1a851","Type":"ContainerStarted","Data":"4acb8eacddb87382680a16e85874618ffc8a754499672d2e756e4f77c3eda5db"}
Sep 14 04:52:49.611504 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:49.611463    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts" event={"ID":"490fd034-c3ef-4dbc-903d-f63015f1a851","Type":"ContainerStarted","Data":"7e061ab8590fad803dac44940af5afe934a54d502b3798deefda041564f71bd6"}
Sep 14 04:52:49.611504 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:49.611507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts" event={"ID":"490fd034-c3ef-4dbc-903d-f63015f1a851","Type":"ContainerStarted","Data":"d0e6134e96a15033a6b66ad4199f7e3f6a73e7d6530f56ce57149052e690c83c"}
Sep 14 04:52:49.611987 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:49.611520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts" event={"ID":"490fd034-c3ef-4dbc-903d-f63015f1a851","Type":"ContainerStarted","Data":"206e77f8d650790a4b6583aff09ae72284059d07a7c858c2fcc651e838ea4202"}
Sep 14 04:52:49.639534 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:49.639478    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bbb7db775-xx6ts" podStartSLOduration=1.945193069 podStartE2EDuration="5.639461922s" totalImagesPullingTime="3.694268853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:52:49.637856511 +0000 UTC m=+363.999808954" watchObservedRunningTime="2026-09-14 04:52:49.639461922 +0000 UTC m=+364.001414404"
Sep 14 04:52:59.627187 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:59.627115    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78fb7cf45d-mnzbp" podUID="b96f9765-f305-4a02-85e7-38c15308d2c5" containerName="console" containerID="cri-o://42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5" gracePeriod=15
Sep 14 04:52:59.878744 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:52:59.878662    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:53:00.027473 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027679 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027476    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027679 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027590    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027679 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027638    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027847 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027682    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027847 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027726    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxcs9\" (UniqueName: \"kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027847 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027760    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle\") pod \"b96f9765-f305-4a02-85e7-38c15308d2c5\" (UID: \"b96f9765-f305-4a02-85e7-38c15308d2c5\") "
Sep 14 04:53:00.027847 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.027825    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:53:00.028067 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.028030    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-oauth-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.028112 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.028078    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config" (OuterVolumeSpecName: "console-config") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:53:00.028161 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.028136    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:53:00.028161 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.028134    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca" (OuterVolumeSpecName: "service-ca") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:53:00.029802 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.029775    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:53:00.029914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.029806    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:53:00.029914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.029858    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9" (OuterVolumeSpecName: "kube-api-access-hxcs9") pod "b96f9765-f305-4a02-85e7-38c15308d2c5" (UID: "b96f9765-f305-4a02-85e7-38c15308d2c5"). InnerVolumeSpecName "kube-api-access-hxcs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129169    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-oauth-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129204    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-console-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129213    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129223    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxcs9\" (UniqueName: \"kubernetes.io/projected/b96f9765-f305-4a02-85e7-38c15308d2c5-kube-api-access-hxcs9\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129232    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b96f9765-f305-4a02-85e7-38c15308d2c5-trusted-ca-bundle\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.129266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.129241    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b96f9765-f305-4a02-85e7-38c15308d2c5-console-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:53:00.654749 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.654711    2571 generic.go:356] "Generic (PLEG): container finished" podID="b96f9765-f305-4a02-85e7-38c15308d2c5" containerID="42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5" exitCode=0
Sep 14 04:53:00.655192 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.654757    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78fb7cf45d-mnzbp"
Sep 14 04:53:00.655192 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.654802    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fb7cf45d-mnzbp" event={"ID":"b96f9765-f305-4a02-85e7-38c15308d2c5","Type":"ContainerDied","Data":"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"}
Sep 14 04:53:00.655192 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.654849    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78fb7cf45d-mnzbp" event={"ID":"b96f9765-f305-4a02-85e7-38c15308d2c5","Type":"ContainerDied","Data":"66eab0ff294c8e0f6804d5fcd68e01763e083da976a234fca668fee8e9afb61f"}
Sep 14 04:53:00.655192 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.654866    2571 scope.go:117] "RemoveContainer" containerID="42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"
Sep 14 04:53:00.663326 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.663304    2571 scope.go:117] "RemoveContainer" containerID="42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"
Sep 14 04:53:00.663654 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:53:00.663632    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5\": container with ID starting with 42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5 not found: ID does not exist" containerID="42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"
Sep 14 04:53:00.663701 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.663667    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5"} err="failed to get container status \"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5\": rpc error: code = NotFound desc = could not find container \"42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5\": container with ID starting with 42caf953a3dc94f73a619994720f4a47257d12c7a4e87484bcaf65496c7ed2c5 not found: ID does not exist"
Sep 14 04:53:00.674986 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.674959    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:53:00.679613 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:00.679590    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78fb7cf45d-mnzbp"]
Sep 14 04:53:02.192725 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:02.192689    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b96f9765-f305-4a02-85e7-38c15308d2c5" path="/var/lib/kubelet/pods/b96f9765-f305-4a02-85e7-38c15308d2c5/volumes"
Sep 14 04:53:22.777628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:22.777593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:22.878242 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:22.878202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:54.449325 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:54.449290    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:56.036423 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:56.036389    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:58.061461 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:58.061397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:58.121254 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:58.121217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:59.353169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:59.353131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:59.468193 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:59.468153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:59.782544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:59.782512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:53:59.869554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:53:59.869500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:02.112387 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:02.112339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:02.318767 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:02.318736    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:04.860824 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:04.860785    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:04.966240 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:04.966212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:09.757518 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:09.757425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:13.145965 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:13.145908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:13.242818 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:13.242788    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:14.830894 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:14.830858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:14.923205 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:14.923170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:15.360434 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.360399    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"]
Sep 14 04:54:15.360708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.360695    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b96f9765-f305-4a02-85e7-38c15308d2c5" containerName="console"
Sep 14 04:54:15.360708 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.360709    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b96f9765-f305-4a02-85e7-38c15308d2c5" containerName="console"
Sep 14 04:54:15.360800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.360762    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b96f9765-f305-4a02-85e7-38c15308d2c5" containerName="console"
Sep 14 04:54:15.363604 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.363586    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.369112 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.369087    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 04:54:15.369264 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.369222    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5bq22\""
Sep 14 04:54:15.369264 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.369251    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 04:54:15.387226 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.387198    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"]
Sep 14 04:54:15.518234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.518189    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5vgm\" (UniqueName: \"kubernetes.io/projected/e2ca4591-bf29-4a32-9516-ffa8c806b289-kube-api-access-k5vgm\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.518454 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.518272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2ca4591-bf29-4a32-9516-ffa8c806b289-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.619721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.619632    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5vgm\" (UniqueName: \"kubernetes.io/projected/e2ca4591-bf29-4a32-9516-ffa8c806b289-kube-api-access-k5vgm\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.619721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.619684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2ca4591-bf29-4a32-9516-ffa8c806b289-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.620146 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.620128    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2ca4591-bf29-4a32-9516-ffa8c806b289-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.629889 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.629849    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5vgm\" (UniqueName: \"kubernetes.io/projected/e2ca4591-bf29-4a32-9516-ffa8c806b289-kube-api-access-k5vgm\") pod \"openshift-lws-operator-fd8ccff4c-85trj\" (UID: \"e2ca4591-bf29-4a32-9516-ffa8c806b289\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.672878 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.672836    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"
Sep 14 04:54:15.965762 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:15.965727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj"]
Sep 14 04:54:15.970271 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:54:15.970245    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2ca4591_bf29_4a32_9516_ffa8c806b289.slice/crio-5c172ce92614feee643a368814a17cb4f6a9e19eb16e34985181a934a4fa9ff0 WatchSource:0}: Error finding container 5c172ce92614feee643a368814a17cb4f6a9e19eb16e34985181a934a4fa9ff0: Status 404 returned error can't find the container with id 5c172ce92614feee643a368814a17cb4f6a9e19eb16e34985181a934a4fa9ff0
Sep 14 04:54:16.957523 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:16.957473    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj" event={"ID":"e2ca4591-bf29-4a32-9516-ffa8c806b289","Type":"ContainerStarted","Data":"5c172ce92614feee643a368814a17cb4f6a9e19eb16e34985181a934a4fa9ff0"}
Sep 14 04:54:18.967484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:18.967443    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj" event={"ID":"e2ca4591-bf29-4a32-9516-ffa8c806b289","Type":"ContainerStarted","Data":"646394a7901807e8f8da63b11c4701ffdbbac9ce2cab060f736eb60bb6754fef"}
Sep 14 04:54:18.987003 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:18.986947    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85trj" podStartSLOduration=1.263068865 podStartE2EDuration="3.98691402s" totalImagesPullingTime="2.723845155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:54:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:54:18.9861196 +0000 UTC m=+453.348072043" watchObservedRunningTime="2026-09-14 04:54:18.98691402 +0000 UTC m=+453.348866462"
Sep 14 04:54:19.663448 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:19.663410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:19.767027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:19.766981    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:22.534225 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:22.534187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:33.169895 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:33.169851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:33.244368 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:33.244331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:33.643919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:33.643883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:33.892361 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:33.892321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:33.996310 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:33.996281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.058496 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.058455    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"]
Sep 14 04:54:34.062031 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.062008    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.065161 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.065136    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 04:54:34.065305 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.065246    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 04:54:34.065305 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.065249    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q4vrb\""
Sep 14 04:54:34.066163 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.066137    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 04:54:34.066163 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.066155    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 04:54:34.070950 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.070913    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-manifests\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.071058 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.070963    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.071058 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.070983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmwwz\" (UniqueName: \"kubernetes.io/projected/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-kube-api-access-zmwwz\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.071227 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.071112    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-tmp\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.071227 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.071157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.088366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.088334    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"]
Sep 14 04:54:34.172239 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-tmp\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.172719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172252    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.172719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-manifests\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.172719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172318    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.172719 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmwwz\" (UniqueName: \"kubernetes.io/projected/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-kube-api-access-zmwwz\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.172907 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.172806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-manifests\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.174680 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.174657    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-tmp\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.174861 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.174840    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.174955 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.174919    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.186145 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.186110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.219074 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.219033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmwwz\" (UniqueName: \"kubernetes.io/projected/282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a-kube-api-access-zmwwz\") pod \"opendatahub-operator-controller-manager-78b6456dc5-p42j6\" (UID: \"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.283801 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.283709    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.372510 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.372475    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:34.383746 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.383723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.484070 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.484035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.587222 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.587129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.663015 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.662988    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"]
Sep 14 04:54:34.666054 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:54:34.665993    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282ec9ea_0c92_4ed7_b8b5_7c7af2c34f9a.slice/crio-85967af22b0a41a3d004fcb80573509283e8f2b473a0298589acc240d33467e8 WatchSource:0}: Error finding container 85967af22b0a41a3d004fcb80573509283e8f2b473a0298589acc240d33467e8: Status 404 returned error can't find the container with id 85967af22b0a41a3d004fcb80573509283e8f2b473a0298589acc240d33467e8
Sep 14 04:54:34.698165 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.698130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:34.807682 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:34.807649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:35.031680 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:35.031645    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6" event={"ID":"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a","Type":"ContainerStarted","Data":"85967af22b0a41a3d004fcb80573509283e8f2b473a0298589acc240d33467e8"}
Sep 14 04:54:36.738189 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.738154    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cfdd6d869-j8hf7"]
Sep 14 04:54:36.743244 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.743158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.753758 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.753727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cfdd6d869-j8hf7"]
Sep 14 04:54:36.788234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.788195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:36.794628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-console-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.794628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-service-ca\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.794628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-oauth-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.794628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.794628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb4xm\" (UniqueName: \"kubernetes.io/projected/186ec657-2c10-4eec-a544-323a1b0b673c-kube-api-access-lb4xm\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.795094 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794667    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-trusted-ca-bundle\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.795094 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.794702    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-oauth-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-trusted-ca-bundle\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-oauth-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-console-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895473    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-service-ca\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-oauth-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895571 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895561    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.895844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.895582    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb4xm\" (UniqueName: \"kubernetes.io/projected/186ec657-2c10-4eec-a544-323a1b0b673c-kube-api-access-lb4xm\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.896318 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.896285    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-oauth-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.896466 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.896285    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-console-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.896466 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.896425    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-trusted-ca-bundle\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.896646 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.896472    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/186ec657-2c10-4eec-a544-323a1b0b673c-service-ca\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.898048 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.897989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:36.899087 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.899061    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-serving-cert\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.899087 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.899075    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/186ec657-2c10-4eec-a544-323a1b0b673c-console-oauth-config\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:36.905662 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:36.905640    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb4xm\" (UniqueName: \"kubernetes.io/projected/186ec657-2c10-4eec-a544-323a1b0b673c-kube-api-access-lb4xm\") pod \"console-7cfdd6d869-j8hf7\" (UID: \"186ec657-2c10-4eec-a544-323a1b0b673c\") " pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:37.001677 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:37.001582    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:37.057701 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:37.057651    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:37.531260 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:37.531205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:37.800419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:37.800363    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cfdd6d869-j8hf7"]
Sep 14 04:54:37.802670 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:54:37.802639    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186ec657_2c10_4eec_a544_323a1b0b673c.slice/crio-076b4c91460bcaa94d2253bda5629fb95683d3a58df51b3d5900b5fabe69ec07 WatchSource:0}: Error finding container 076b4c91460bcaa94d2253bda5629fb95683d3a58df51b3d5900b5fabe69ec07: Status 404 returned error can't find the container with id 076b4c91460bcaa94d2253bda5629fb95683d3a58df51b3d5900b5fabe69ec07
Sep 14 04:54:38.050912 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:38.050810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6" event={"ID":"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a","Type":"ContainerStarted","Data":"aa71e9d02acd5f8cb0b3b948f5b9ca6243f268c4ceb7dac12606c1caa4eca02f"}
Sep 14 04:54:38.052456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:38.052435    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfdd6d869-j8hf7" event={"ID":"186ec657-2c10-4eec-a544-323a1b0b673c","Type":"ContainerStarted","Data":"e09137a9213aa9ba80022276c59d9c7261d9a6abfd34dcddffe452af5e729750"}
Sep 14 04:54:38.052561 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:38.052473    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cfdd6d869-j8hf7" event={"ID":"186ec657-2c10-4eec-a544-323a1b0b673c","Type":"ContainerStarted","Data":"076b4c91460bcaa94d2253bda5629fb95683d3a58df51b3d5900b5fabe69ec07"}
Sep 14 04:54:38.097035 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:38.096968    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cfdd6d869-j8hf7" podStartSLOduration=2.096916093 podStartE2EDuration="2.096916093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:54:38.096481802 +0000 UTC m=+472.458434246" watchObservedRunningTime="2026-09-14 04:54:38.096916093 +0000 UTC m=+472.458868544"
Sep 14 04:54:39.058347 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:39.058312    2571 generic.go:356] "Generic (PLEG): container finished" podID="282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a" containerID="aa71e9d02acd5f8cb0b3b948f5b9ca6243f268c4ceb7dac12606c1caa4eca02f" exitCode=0
Sep 14 04:54:39.058761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:39.058399    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6" event={"ID":"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a","Type":"ContainerDied","Data":"aa71e9d02acd5f8cb0b3b948f5b9ca6243f268c4ceb7dac12606c1caa4eca02f"}
Sep 14 04:54:40.066268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:40.066229    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6" event={"ID":"282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a","Type":"ContainerStarted","Data":"71b139ecc831c4d097b3b8aa4cd2598cd4d82a37d3cc3778b7869297e565fbe9"}
Sep 14 04:54:40.066800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:40.066387    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:40.105162 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:40.105094    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6" podStartSLOduration=1.851531981 podStartE2EDuration="6.105070283s" totalImagesPullingTime="3.234634871s" totalInitContainerRuntime="1.018903431s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:54:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:54:40.103548549 +0000 UTC m=+474.465500993" watchObservedRunningTime="2026-09-14 04:54:40.105070283 +0000 UTC m=+474.467022728"
Sep 14 04:54:42.833192 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:42.833157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:42.929238 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:42.929210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:47.057912 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.057874    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:47.058443 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.057947    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:47.063046 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.063011    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:47.100290 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.100260    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cfdd6d869-j8hf7"
Sep 14 04:54:47.250541 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.250507    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:54:47.423419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.423326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:47.524409 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.524374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:47.624813 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:47.624773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:51.073636 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:51.073599    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78b6456dc5-p42j6"
Sep 14 04:54:53.780466 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:53.780418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:53.926556 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:53.926518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:54.657536 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.657499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:54.830056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.830023    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-96745c74f-dttph"]
Sep 14 04:54:54.976476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.976445    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-96745c74f-dttph"]
Sep 14 04:54:54.976642 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.976570    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:54.994293 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.994255    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 04:54:54.994493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.994452    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 04:54:54.995498 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.995477    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 04:54:54.995626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.995479    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tznbv\""
Sep 14 04:54:54.995626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:54.995480    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 04:54:55.047772 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.047727    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n6cv\" (UniqueName: \"kubernetes.io/projected/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-kube-api-access-2n6cv\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.047975 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.047792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tls-certs\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.047975 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.047888    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tmp\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.148639 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.148594    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tls-certs\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.148845 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.148653    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tmp\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.148845 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.148710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n6cv\" (UniqueName: \"kubernetes.io/projected/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-kube-api-access-2n6cv\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.151197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.151159    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tmp\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.151396 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.151377    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-tls-certs\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.170715 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.170679    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n6cv\" (UniqueName: \"kubernetes.io/projected/82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04-kube-api-access-2n6cv\") pod \"kube-auth-proxy-96745c74f-dttph\" (UID: \"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04\") " pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.182084 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.182048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:55.286602 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.286509    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph"
Sep 14 04:54:55.592523 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:55.592378    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-96745c74f-dttph"]
Sep 14 04:54:55.594789 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:54:55.594765    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82aea7cf_0a7b_4c24_ad00_c48a5bd1bf04.slice/crio-a79723a1c52b4564760ead1ba9de39bd43ea31eed6630f19f0f0a1917525e3da WatchSource:0}: Error finding container a79723a1c52b4564760ead1ba9de39bd43ea31eed6630f19f0f0a1917525e3da: Status 404 returned error can't find the container with id a79723a1c52b4564760ead1ba9de39bd43ea31eed6630f19f0f0a1917525e3da
Sep 14 04:54:56.140604 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:56.140565    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph" event={"ID":"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04","Type":"ContainerStarted","Data":"a79723a1c52b4564760ead1ba9de39bd43ea31eed6630f19f0f0a1917525e3da"}
Sep 14 04:54:57.145700 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.145519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:57.190101 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.190060    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:54:57.193628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.193595    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.198218 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.198189    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 04:54:57.198380 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.198268    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v8cmr\""
Sep 14 04:54:57.198605 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.198521    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 04:54:57.198790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.198610    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 04:54:57.198887 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.198839    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 04:54:57.224873 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.224839    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:54:57.266687 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.266651    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:57.268666 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.268808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.268808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268711    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.268900 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lf2\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.268900 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268848    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.269022 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268921    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.269022 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.268982    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.369923 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.369884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lf2\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.369959    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.369996    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.370024    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.370064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370134 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.370110    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370407 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.370146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.370794 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.370689    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.373468 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.373443    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.373808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.373787    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.373898 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.373833    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.374249 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.374224    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.382795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.382748    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.383721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.383694    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lf2\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2\") pod \"istiod-openshift-gateway-7b7995d5cd-q7src\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:57.507968 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:57.507912    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:54:58.425834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:58.425806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:54:58.428592 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:54:58.428564    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6727bdd_8cb1_4c27_98e2_405770c9613f.slice/crio-c09c144f54058df272b3438816970f0a9db61d126dc7293a4406af840c0783ac WatchSource:0}: Error finding container c09c144f54058df272b3438816970f0a9db61d126dc7293a4406af840c0783ac: Status 404 returned error can't find the container with id c09c144f54058df272b3438816970f0a9db61d126dc7293a4406af840c0783ac
Sep 14 04:54:58.698517 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:58.698440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:54:59.158000 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:59.157959    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph" event={"ID":"82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04","Type":"ContainerStarted","Data":"d931ffa8b1ec6beae2e269593f7b5b3b7b57033597d1a41d3630fb13e17d81fe"}
Sep 14 04:54:59.159376 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:59.159347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" event={"ID":"f6727bdd-8cb1-4c27-98e2-405770c9613f","Type":"ContainerStarted","Data":"c09c144f54058df272b3438816970f0a9db61d126dc7293a4406af840c0783ac"}
Sep 14 04:54:59.187608 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:54:59.187552    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-96745c74f-dttph" podStartSLOduration=2.571165272 podStartE2EDuration="5.187530266s" totalImagesPullingTime="2.616364994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:54:59.181693607 +0000 UTC m=+493.543646049" watchObservedRunningTime="2026-09-14 04:54:59.187530266 +0000 UTC m=+493.549482708"
Sep 14 04:55:01.032970 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.032911    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 04:55:01.033351 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.033015    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 04:55:01.171414 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.171371    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" event={"ID":"f6727bdd-8cb1-4c27-98e2-405770c9613f","Type":"ContainerStarted","Data":"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"}
Sep 14 04:55:01.171597 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.171459    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:55:01.205628 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.205571    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" podStartSLOduration=1.603809812 podStartE2EDuration="4.205554023s" totalImagesPullingTime="2.601744211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:55:01.203241756 +0000 UTC m=+495.565194198" watchObservedRunningTime="2026-09-14 04:55:01.205554023 +0000 UTC m=+495.567506466"
Sep 14 04:55:01.840097 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.840060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:01.938746 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:01.938692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:02.104684 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:02.104600    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:02.177947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:02.177908    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-q7src container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 04:55:02.178110 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:02.177981    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 04:55:02.367015 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:02.366908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:02.469463 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:02.469433    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:04.557541 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.557509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:04.655528 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.655490    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:04.760384 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.760347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:04.932484 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.932397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:04.936864 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.936832    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5wwgd"]
Sep 14 04:55:04.941186 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.941165    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:04.945623 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.945601    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4phnn\""
Sep 14 04:55:04.946470 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.946446    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 14 04:55:04.957798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:04.957769    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5wwgd"]
Sep 14 04:55:05.037037 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.036997    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxkq\" (UniqueName: \"kubernetes.io/projected/8ef3c739-010f-48a7-a2a4-318b70c37d4d-kube-api-access-2wxkq\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.037212 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.037080    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ef3c739-010f-48a7-a2a4-318b70c37d4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.136283 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.136255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:05.138429 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.138401    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxkq\" (UniqueName: \"kubernetes.io/projected/8ef3c739-010f-48a7-a2a4-318b70c37d4d-kube-api-access-2wxkq\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.138555 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.138446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ef3c739-010f-48a7-a2a4-318b70c37d4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.141093 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.141064    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ef3c739-010f-48a7-a2a4-318b70c37d4d-tls-certs\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.147538 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.147515    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxkq\" (UniqueName: \"kubernetes.io/projected/8ef3c739-010f-48a7-a2a4-318b70c37d4d-kube-api-access-2wxkq\") pod \"model-serving-api-54d89bd5fc-5wwgd\" (UID: \"8ef3c739-010f-48a7-a2a4-318b70c37d4d\") " pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.178169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.178142    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:55:05.251994 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.251959    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:05.529766 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:05.529623    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5wwgd"]
Sep 14 04:55:05.532551 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:55:05.532511    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef3c739_010f_48a7_a2a4_318b70c37d4d.slice/crio-18601b294190307cb708f80ebfc5424ca88606d10cba55a226a125525dd922b6 WatchSource:0}: Error finding container 18601b294190307cb708f80ebfc5424ca88606d10cba55a226a125525dd922b6: Status 404 returned error can't find the container with id 18601b294190307cb708f80ebfc5424ca88606d10cba55a226a125525dd922b6
Sep 14 04:55:06.198282 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:06.198205    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd" event={"ID":"8ef3c739-010f-48a7-a2a4-318b70c37d4d","Type":"ContainerStarted","Data":"18601b294190307cb708f80ebfc5424ca88606d10cba55a226a125525dd922b6"}
Sep 14 04:55:07.205061 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:07.205023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd" event={"ID":"8ef3c739-010f-48a7-a2a4-318b70c37d4d","Type":"ContainerStarted","Data":"c7f320094973c626f54fc26683e57d4d04b6c47f854a21ba6cb0c524ecdb0545"}
Sep 14 04:55:07.205485 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:07.205173    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:07.224844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:07.224791    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd" podStartSLOduration=1.625675227 podStartE2EDuration="3.224768579s" totalImagesPullingTime="1.599093352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:55:07.223398143 +0000 UTC m=+501.585350586" watchObservedRunningTime="2026-09-14 04:55:07.224768579 +0000 UTC m=+501.586721021"
Sep 14 04:55:12.270787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.270738    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-88878b64b-6hnwb" podUID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" containerName="console" containerID="cri-o://5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f" gracePeriod=15
Sep 14 04:55:12.522037 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.521980    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:55:12.614806 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.614761    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.614806 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.614802    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcrxw\" (UniqueName: \"kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615092 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.614840    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615092 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.614874    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615092 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.614924    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615361 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615338    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615373    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config\") pod \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\" (UID: \"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6\") "
Sep 14 04:55:12.615506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615364    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:55:12.615671 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615655    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-oauth-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.615859 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615831    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config" (OuterVolumeSpecName: "console-config") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:55:12.615915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.615875    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:55:12.616029 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.616006    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:55:12.617538 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.617501    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:55:12.617728 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.617708    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:55:12.617954 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.617910    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw" (OuterVolumeSpecName: "kube-api-access-pcrxw") pod "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" (UID: "d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6"). InnerVolumeSpecName "kube-api-access-pcrxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:55:12.716410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716365    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-serving-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.716410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716398    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcrxw\" (UniqueName: \"kubernetes.io/projected/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-kube-api-access-pcrxw\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.716410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716409    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-oauth-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.716410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716418    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.716681 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716426    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-trusted-ca-bundle\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:12.716681 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:12.716436    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6-console-config\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:13.232053 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.232020    2571 generic.go:356] "Generic (PLEG): container finished" podID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" containerID="5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f" exitCode=0
Sep 14 04:55:13.232265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.232080    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-88878b64b-6hnwb"
Sep 14 04:55:13.232265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.232085    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88878b64b-6hnwb" event={"ID":"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6","Type":"ContainerDied","Data":"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"}
Sep 14 04:55:13.232265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.232215    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-88878b64b-6hnwb" event={"ID":"d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6","Type":"ContainerDied","Data":"2a0b7161b6a40157e3138a680a59dd959cb768c283feeb3d11fb1ebdf8f63676"}
Sep 14 04:55:13.232265 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.232248    2571 scope.go:117] "RemoveContainer" containerID="5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"
Sep 14 04:55:13.241376 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.241360    2571 scope.go:117] "RemoveContainer" containerID="5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"
Sep 14 04:55:13.241645 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:13.241627    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f\": container with ID starting with 5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f not found: ID does not exist" containerID="5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"
Sep 14 04:55:13.241713 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.241654    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f"} err="failed to get container status \"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f\": rpc error: code = NotFound desc = could not find container \"5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f\": container with ID starting with 5edf7215f6830231937987fb9a15350a532b5ee096526e779a98a0b8c8d37f4f not found: ID does not exist"
Sep 14 04:55:13.323669 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.323636    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:55:13.324963 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:13.324938    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-88878b64b-6hnwb"]
Sep 14 04:55:14.193815 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:14.193777    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" path="/var/lib/kubelet/pods/d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6/volumes"
Sep 14 04:55:15.391151 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.391119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:15.505527 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.505494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:15.517032 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.516992    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"]
Sep 14 04:55:15.517410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.517390    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" containerName="console"
Sep 14 04:55:15.517497 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.517413    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" containerName="console"
Sep 14 04:55:15.517549 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.517524    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6eecf13-a29f-4f5d-bd2d-491d0b7d83c6" containerName="console"
Sep 14 04:55:15.522158 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.522134    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.524914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.524891    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 04:55:15.525056 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.524894    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dr7rp\""
Sep 14 04:55:15.539493 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.539455    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"]
Sep 14 04:55:15.555419 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.555387    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"]
Sep 14 04:55:15.558677 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.558656    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.572100 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.566368    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9rt4m\""
Sep 14 04:55:15.572100 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.566408    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 14 04:55:15.586405 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.586351    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"]
Sep 14 04:55:15.606973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.606911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:15.642410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.642303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2zgw\" (UniqueName: \"kubernetes.io/projected/c16eebb5-a63b-4538-bf06-045b63b7d9db-kube-api-access-z2zgw\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.642410 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.642400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt4dv\" (UniqueName: \"kubernetes.io/projected/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-kube-api-access-tt4dv\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.642609 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.642422    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.642609 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.642458    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.743683 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.743634    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt4dv\" (UniqueName: \"kubernetes.io/projected/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-kube-api-access-tt4dv\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.743683 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.743683    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.743924 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:15.743784    2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 14 04:55:15.743924 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.743797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.743924 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:15.743848    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert podName:50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb nodeName:}" failed. No retries permitted until 2026-09-14 04:55:16.243830593 +0000 UTC m=+510.605783014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert") pod "kserve-controller-manager-d6f4b65b8-5jl2t" (UID: "50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb") : secret "kserve-webhook-server-cert" not found
Sep 14 04:55:15.743924 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.743882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2zgw\" (UniqueName: \"kubernetes.io/projected/c16eebb5-a63b-4538-bf06-045b63b7d9db-kube-api-access-z2zgw\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.744168 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:15.743884    2571 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 14 04:55:15.744168 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:15.744027    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert podName:c16eebb5-a63b-4538-bf06-045b63b7d9db nodeName:}" failed. No retries permitted until 2026-09-14 04:55:16.244012538 +0000 UTC m=+510.605964967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert") pod "llmisvc-controller-manager-777c6bd7d5-ttvrr" (UID: "c16eebb5-a63b-4538-bf06-045b63b7d9db") : secret "llmisvc-webhook-server-cert" not found
Sep 14 04:55:15.754862 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.754831    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2zgw\" (UniqueName: \"kubernetes.io/projected/c16eebb5-a63b-4538-bf06-045b63b7d9db-kube-api-access-z2zgw\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:15.755974 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.755923    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt4dv\" (UniqueName: \"kubernetes.io/projected/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-kube-api-access-tt4dv\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:15.765151 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.765115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:15.883379 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.883339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:15.989082 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:15.989033    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:16.069304 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.069270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:16.248712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.248631    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:16.248712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.248666    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:16.251334 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.251307    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c16eebb5-a63b-4538-bf06-045b63b7d9db-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ttvrr\" (UID: \"c16eebb5-a63b-4538-bf06-045b63b7d9db\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:16.251456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.251436    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb-cert\") pod \"kserve-controller-manager-d6f4b65b8-5jl2t\" (UID: \"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:16.255384 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.255355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:55:16.432420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.432370    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:16.473002 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.472917    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:16.737231 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.737127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"]
Sep 14 04:55:16.739556 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:55:16.739523    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c9c4c1_a7b2_453a_9c5d_69497bc0b5fb.slice/crio-48cce519f549ce60b6f59172bc1235bbef35e549fbf45f3f42b808258884e295 WatchSource:0}: Error finding container 48cce519f549ce60b6f59172bc1235bbef35e549fbf45f3f42b808258884e295: Status 404 returned error can't find the container with id 48cce519f549ce60b6f59172bc1235bbef35e549fbf45f3f42b808258884e295
Sep 14 04:55:16.759291 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.759265    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"]
Sep 14 04:55:16.761622 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:55:16.761588    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc16eebb5_a63b_4538_bf06_045b63b7d9db.slice/crio-00b5149621313b340e865f748981c147e146e5fce52f981607b45ea446a3cb78 WatchSource:0}: Error finding container 00b5149621313b340e865f748981c147e146e5fce52f981607b45ea446a3cb78: Status 404 returned error can't find the container with id 00b5149621313b340e865f748981c147e146e5fce52f981607b45ea446a3cb78
Sep 14 04:55:16.830958 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.830843    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:55:16.831147 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:16.831066    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerName="discovery" containerID="cri-o://66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f" gracePeriod=30
Sep 14 04:55:17.089806 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.089740    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:55:17.253716 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.253686    2571 generic.go:356] "Generic (PLEG): container finished" podID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerID="66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f" exitCode=0
Sep 14 04:55:17.253908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.253766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" event={"ID":"f6727bdd-8cb1-4c27-98e2-405770c9613f","Type":"ContainerDied","Data":"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"}
Sep 14 04:55:17.253908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.253792    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"
Sep 14 04:55:17.253908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.253832    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src" event={"ID":"f6727bdd-8cb1-4c27-98e2-405770c9613f","Type":"ContainerDied","Data":"c09c144f54058df272b3438816970f0a9db61d126dc7293a4406af840c0783ac"}
Sep 14 04:55:17.253908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.253863    2571 scope.go:117] "RemoveContainer" containerID="66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"
Sep 14 04:55:17.255209 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.255186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr" event={"ID":"c16eebb5-a63b-4538-bf06-045b63b7d9db","Type":"ContainerStarted","Data":"00b5149621313b340e865f748981c147e146e5fce52f981607b45ea446a3cb78"}
Sep 14 04:55:17.255998 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.255977    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256124 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256038    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256124 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256060    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256124 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256105    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256241 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256131    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256241 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256177    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25lf2\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256306 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256245    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig\") pod \"f6727bdd-8cb1-4c27-98e2-405770c9613f\" (UID: \"f6727bdd-8cb1-4c27-98e2-405770c9613f\") "
Sep 14 04:55:17.256455 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256435    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:55:17.256519 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.256503    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-ca-configmap\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.257614 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.257541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t" event={"ID":"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb","Type":"ContainerStarted","Data":"48cce519f549ce60b6f59172bc1235bbef35e549fbf45f3f42b808258884e295"}
Sep 14 04:55:17.259147 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.259101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2" (OuterVolumeSpecName: "kube-api-access-25lf2") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "kube-api-access-25lf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:55:17.259147 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.259101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:55:17.259702 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.259676    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:55:17.259787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.259763    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs" (OuterVolumeSpecName: "local-certs") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:55:17.260132 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.260101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:55:17.260255 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.260130    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts" (OuterVolumeSpecName: "cacerts") pod "f6727bdd-8cb1-4c27-98e2-405770c9613f" (UID: "f6727bdd-8cb1-4c27-98e2-405770c9613f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:55:17.297544 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.297521    2571 scope.go:117] "RemoveContainer" containerID="66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"
Sep 14 04:55:17.297902 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:55:17.297877    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f\": container with ID starting with 66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f not found: ID does not exist" containerID="66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"
Sep 14 04:55:17.297984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.297913    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f"} err="failed to get container status \"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f\": rpc error: code = NotFound desc = could not find container \"66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f\": container with ID starting with 66c7eaf696c74733ab57ab404821e1fc6bdeb6188fdd7e8bbf6c99ab0e25836f not found: ID does not exist"
Sep 14 04:55:17.357656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357570    2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-cacerts\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.357656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357612    2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6727bdd-8cb1-4c27-98e2-405770c9613f-local-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.357656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357629    2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-csr-dns-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.357656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357642    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25lf2\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-kube-api-access-25lf2\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.357656 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357655    2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-kubeconfig\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.357955 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.357668    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6727bdd-8cb1-4c27-98e2-405770c9613f-istio-token\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:55:17.584372 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.584335    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:55:17.587323 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:17.587291    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-q7src"]
Sep 14 04:55:18.196469 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:18.196380    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" path="/var/lib/kubelet/pods/f6727bdd-8cb1-4c27-98e2-405770c9613f/volumes"
Sep 14 04:55:18.215148 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:18.215115    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5wwgd"
Sep 14 04:55:20.275861 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.275814    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t" event={"ID":"50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb","Type":"ContainerStarted","Data":"b817d35c7ded8041304c4b8da603f0f94f1f10576f0a95ca551e86a1840c1394"}
Sep 14 04:55:20.276308 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.275886    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:55:20.277502 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.277477    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr" event={"ID":"c16eebb5-a63b-4538-bf06-045b63b7d9db","Type":"ContainerStarted","Data":"64f10e34ad766d08c70cdb2367728c1db360b69489881a8f1542ce14ebea42ce"}
Sep 14 04:55:20.277609 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.277603    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:20.294503 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.294450    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t" podStartSLOduration=2.406085942 podStartE2EDuration="5.294433244s" totalImagesPullingTime="2.888347302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:55:20.293836308 +0000 UTC m=+514.655788751" watchObservedRunningTime="2026-09-14 04:55:20.294433244 +0000 UTC m=+514.656385686"
Sep 14 04:55:20.312027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:20.311973    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr" podStartSLOduration=2.440881228 podStartE2EDuration="5.311958248s" totalImagesPullingTime="2.87107702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:55:20.310157527 +0000 UTC m=+514.672109970" watchObservedRunningTime="2026-09-14 04:55:20.311958248 +0000 UTC m=+514.673910729"
Sep 14 04:55:51.284915 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:51.284881    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ttvrr"
Sep 14 04:55:51.287848 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:55:51.287823    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5jl2t"
Sep 14 04:56:05.825603 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:05.825565    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:05.917196 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:05.917160    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:18.683259 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:18.683199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:19.486900 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:19.486871    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:21.745000 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.744958    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:21.976519 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.976483    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2hfcm"]
Sep 14 04:56:21.976848 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.976835    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerName="discovery"
Sep 14 04:56:21.976895 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.976851    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerName="discovery"
Sep 14 04:56:21.976957 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.976916    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6727bdd-8cb1-4c27-98e2-405770c9613f" containerName="discovery"
Sep 14 04:56:21.979989 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.979970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:21.982866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.982834    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 04:56:21.982866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.982843    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rwzrz\""
Sep 14 04:56:21.983610 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.983595    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 04:56:21.992027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:21.992002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2hfcm"]
Sep 14 04:56:22.021111 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.021016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl8r4\" (UniqueName: \"kubernetes.io/projected/57b59a14-4a57-49de-bc36-9888741c0dcc-kube-api-access-gl8r4\") pod \"authorino-operator-76b478bd78-2hfcm\" (UID: \"57b59a14-4a57-49de-bc36-9888741c0dcc\") " pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:22.122314 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.122274    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl8r4\" (UniqueName: \"kubernetes.io/projected/57b59a14-4a57-49de-bc36-9888741c0dcc-kube-api-access-gl8r4\") pod \"authorino-operator-76b478bd78-2hfcm\" (UID: \"57b59a14-4a57-49de-bc36-9888741c0dcc\") " pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:22.159037 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.158993    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl8r4\" (UniqueName: \"kubernetes.io/projected/57b59a14-4a57-49de-bc36-9888741c0dcc-kube-api-access-gl8r4\") pod \"authorino-operator-76b478bd78-2hfcm\" (UID: \"57b59a14-4a57-49de-bc36-9888741c0dcc\") " pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:22.290384 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.290289    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:22.390688 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.390653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:22.507539 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.507494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:22.566691 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:22.566587    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2hfcm"]
Sep 14 04:56:22.569567 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:22.569534    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b59a14_4a57_49de_bc36_9888741c0dcc.slice/crio-a56ec1c0611aca12102d7ba9e270d08549e36d133fd396fa1686558aac111160 WatchSource:0}: Error finding container a56ec1c0611aca12102d7ba9e270d08549e36d133fd396fa1686558aac111160: Status 404 returned error can't find the container with id a56ec1c0611aca12102d7ba9e270d08549e36d133fd396fa1686558aac111160
Sep 14 04:56:23.092517 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:23.092479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:23.524907 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:23.524808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:23.578118 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:23.578079    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm" event={"ID":"57b59a14-4a57-49de-bc36-9888741c0dcc","Type":"ContainerStarted","Data":"a56ec1c0611aca12102d7ba9e270d08549e36d133fd396fa1686558aac111160"}
Sep 14 04:56:24.822144 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:24.822105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:24.929297 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:24.929258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:25.590032 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:25.589997    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm" event={"ID":"57b59a14-4a57-49de-bc36-9888741c0dcc","Type":"ContainerStarted","Data":"34d9aaaf662177a5eec8db0c2de3a65cb0742109b3aced2163ef19dde44a5c86"}
Sep 14 04:56:25.590209 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:25.590092    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:25.610515 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:25.610454    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm" podStartSLOduration=2.18299361 podStartE2EDuration="4.610436843s" totalImagesPullingTime="2.427443233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:25.609261633 +0000 UTC m=+579.971214077" watchObservedRunningTime="2026-09-14 04:56:25.610436843 +0000 UTC m=+579.972389285"
Sep 14 04:56:26.374411 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.374374    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"]
Sep 14 04:56:26.377808 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.377787    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.382296 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.382266    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-29dd4\""
Sep 14 04:56:26.414524 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.414487    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"]
Sep 14 04:56:26.463480 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.463440    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.463658 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.463501    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk6xj\" (UniqueName: \"kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.564759 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.564719    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk6xj\" (UniqueName: \"kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.564969 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.564825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.565242 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.565223    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.576647 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.576606    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk6xj\" (UniqueName: \"kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.694800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.694708    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:26.917712 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.917672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:26.981761 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:26.981730    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"]
Sep 14 04:56:26.984485 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:26.984457    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4cae6e_e188_40ff_9052_d5398c048575.slice/crio-1eb145781e056af8fbeaf03094b0a8289b28afcbcccfce1c87c2013c2ea3ce2a WatchSource:0}: Error finding container 1eb145781e056af8fbeaf03094b0a8289b28afcbcccfce1c87c2013c2ea3ce2a: Status 404 returned error can't find the container with id 1eb145781e056af8fbeaf03094b0a8289b28afcbcccfce1c87c2013c2ea3ce2a
Sep 14 04:56:27.016558 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:27.016527    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:27.601541 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:27.601505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h" event={"ID":"0c4cae6e-e188-40ff-9052-d5398c048575","Type":"ContainerStarted","Data":"1eb145781e056af8fbeaf03094b0a8289b28afcbcccfce1c87c2013c2ea3ce2a"}
Sep 14 04:56:28.438486 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.438443    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"]
Sep 14 04:56:28.442233 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.442205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:28.445013 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.444986    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 04:56:28.445159 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.445093    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4fhmf\""
Sep 14 04:56:28.458660 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.458631    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"]
Sep 14 04:56:28.484422 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.484385    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvhl4\" (UniqueName: \"kubernetes.io/projected/03f32217-adf1-4e92-82ce-baf9cb82c543-kube-api-access-qvhl4\") pod \"dns-operator-controller-manager-7b84cd7c9c-q2fl6\" (UID: \"03f32217-adf1-4e92-82ce-baf9cb82c543\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:28.585688 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.585633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvhl4\" (UniqueName: \"kubernetes.io/projected/03f32217-adf1-4e92-82ce-baf9cb82c543-kube-api-access-qvhl4\") pod \"dns-operator-controller-manager-7b84cd7c9c-q2fl6\" (UID: \"03f32217-adf1-4e92-82ce-baf9cb82c543\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:28.597086 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.597047    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvhl4\" (UniqueName: \"kubernetes.io/projected/03f32217-adf1-4e92-82ce-baf9cb82c543-kube-api-access-qvhl4\") pod \"dns-operator-controller-manager-7b84cd7c9c-q2fl6\" (UID: \"03f32217-adf1-4e92-82ce-baf9cb82c543\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:28.755739 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:28.755698    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:29.053703 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:29.053617    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"]
Sep 14 04:56:29.056816 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:29.056771    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f32217_adf1_4e92_82ce_baf9cb82c543.slice/crio-f28a9069e3b8ce9884c88d405432f61d23ea1a83045b348d0003bebb65297c4c WatchSource:0}: Error finding container f28a9069e3b8ce9884c88d405432f61d23ea1a83045b348d0003bebb65297c4c: Status 404 returned error can't find the container with id f28a9069e3b8ce9884c88d405432f61d23ea1a83045b348d0003bebb65297c4c
Sep 14 04:56:29.115448 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:29.115412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:29.613787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:29.613747    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6" event={"ID":"03f32217-adf1-4e92-82ce-baf9cb82c543","Type":"ContainerStarted","Data":"f28a9069e3b8ce9884c88d405432f61d23ea1a83045b348d0003bebb65297c4c"}
Sep 14 04:56:30.541359 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.541306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:30.669143 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.669103    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"]
Sep 14 04:56:30.674003 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.673971    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:30.677050 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.677026    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vs8f7\""
Sep 14 04:56:30.686470 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.686439    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"]
Sep 14 04:56:30.808481 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.808442    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d4w6\" (UniqueName: \"kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6\") pod \"limitador-operator-controller-manager-74495c69f8-55xrj\" (UID: \"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:30.909561 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.909520    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d4w6\" (UniqueName: \"kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6\") pod \"limitador-operator-controller-manager-74495c69f8-55xrj\" (UID: \"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:30.919673 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.919640    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d4w6\" (UniqueName: \"kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6\") pod \"limitador-operator-controller-manager-74495c69f8-55xrj\" (UID: \"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:30.986676 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:30.986641    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:31.278570 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:31.278495    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"]
Sep 14 04:56:31.281493 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:31.281464    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12e6ebe_2ac3_409f_9b8d_71daa0ffd6e4.slice/crio-227b1f54dd68a98f590851ea556a1cb5489aba7cc0fc8885a41bc9bd6e89260c WatchSource:0}: Error finding container 227b1f54dd68a98f590851ea556a1cb5489aba7cc0fc8885a41bc9bd6e89260c: Status 404 returned error can't find the container with id 227b1f54dd68a98f590851ea556a1cb5489aba7cc0fc8885a41bc9bd6e89260c
Sep 14 04:56:31.627190 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:31.627147    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h" event={"ID":"0c4cae6e-e188-40ff-9052-d5398c048575","Type":"ContainerStarted","Data":"c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb"}
Sep 14 04:56:31.627702 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:31.627309    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:31.628769 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:31.628738    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj" event={"ID":"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4","Type":"ContainerStarted","Data":"227b1f54dd68a98f590851ea556a1cb5489aba7cc0fc8885a41bc9bd6e89260c"}
Sep 14 04:56:31.654377 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:31.654314    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h" podStartSLOduration=1.859371818 podStartE2EDuration="5.654291741s" totalImagesPullingTime="3.794919923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:31.65140888 +0000 UTC m=+586.013361325" watchObservedRunningTime="2026-09-14 04:56:31.654291741 +0000 UTC m=+586.016244186"
Sep 14 04:56:32.103914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.103877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:32.207553 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.207521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:32.608239 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.608201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:32.634798 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.634762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6" event={"ID":"03f32217-adf1-4e92-82ce-baf9cb82c543","Type":"ContainerStarted","Data":"42ffd058b8d4f311e065e02277edbf45bc37e6eb259e1c0df1504897c5a08261"}
Sep 14 04:56:32.635279 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.635046    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:32.658947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.658875    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6" podStartSLOduration=1.931749538 podStartE2EDuration="4.658856671s" totalImagesPullingTime="2.727107133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:32.657649587 +0000 UTC m=+587.019602045" watchObservedRunningTime="2026-09-14 04:56:32.658856671 +0000 UTC m=+587.020809105"
Sep 14 04:56:32.666909 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:32.666876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:33.739626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.739591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:33.750092 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.750057    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"]
Sep 14 04:56:33.753635 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.753428    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.756146 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.756075    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 14 04:56:33.756146 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.756119    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 14 04:56:33.756332 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.756231    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-74g7q\""
Sep 14 04:56:33.761515 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.761488    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"]
Sep 14 04:56:33.838908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.838872    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06850d28-9a58-4833-9687-30f9f016efe5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.839090 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.838952    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06850d28-9a58-4833-9687-30f9f016efe5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.839166 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.839100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhkpq\" (UniqueName: \"kubernetes.io/projected/06850d28-9a58-4833-9687-30f9f016efe5-kube-api-access-nhkpq\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.939894 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.939855    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06850d28-9a58-4833-9687-30f9f016efe5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.940103 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.939978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhkpq\" (UniqueName: \"kubernetes.io/projected/06850d28-9a58-4833-9687-30f9f016efe5-kube-api-access-nhkpq\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.940103 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.940023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06850d28-9a58-4833-9687-30f9f016efe5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.940562 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.940534    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06850d28-9a58-4833-9687-30f9f016efe5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.942523 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.942500    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/06850d28-9a58-4833-9687-30f9f016efe5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.949368 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.949313    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhkpq\" (UniqueName: \"kubernetes.io/projected/06850d28-9a58-4833-9687-30f9f016efe5-kube-api-access-nhkpq\") pod \"kuadrant-console-plugin-fbdd4f445-k26jx\" (UID: \"06850d28-9a58-4833-9687-30f9f016efe5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:33.985842 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:33.985810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:34.064214 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.064144    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"
Sep 14 04:56:34.372505 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.372410    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx"]
Sep 14 04:56:34.376288 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:34.376258    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06850d28_9a58_4833_9687_30f9f016efe5.slice/crio-18eeab3abe0772d5a9071c6be9b35fbeab247311818fce2ffc0cb69f599cb04a WatchSource:0}: Error finding container 18eeab3abe0772d5a9071c6be9b35fbeab247311818fce2ffc0cb69f599cb04a: Status 404 returned error can't find the container with id 18eeab3abe0772d5a9071c6be9b35fbeab247311818fce2ffc0cb69f599cb04a
Sep 14 04:56:34.647870 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.647782    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj" event={"ID":"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4","Type":"ContainerStarted","Data":"f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9"}
Sep 14 04:56:34.647870 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.647863    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:34.649102 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.649078    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx" event={"ID":"06850d28-9a58-4833-9687-30f9f016efe5","Type":"ContainerStarted","Data":"18eeab3abe0772d5a9071c6be9b35fbeab247311818fce2ffc0cb69f599cb04a"}
Sep 14 04:56:34.666142 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:34.666087    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj" podStartSLOduration=2.195435244 podStartE2EDuration="4.666069323s" totalImagesPullingTime="2.470634079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:34.664998997 +0000 UTC m=+589.026951441" watchObservedRunningTime="2026-09-14 04:56:34.666069323 +0000 UTC m=+589.028021769"
Sep 14 04:56:36.597169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:36.597130    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-2hfcm"
Sep 14 04:56:38.329139 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.329099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:38.397034 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.396989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:38.489838 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.489803    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:38.573674 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.573638    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:38.695442 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.695342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:38.761802 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:38.761763    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:39.686314 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:39.686273    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx" event={"ID":"06850d28-9a58-4833-9687-30f9f016efe5","Type":"ContainerStarted","Data":"b0d7fa5b26e59b5896569fe20305916ee47b1c79ec72d0693bf0042c994c4b39"}
Sep 14 04:56:39.704328 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:39.704272    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-k26jx" podStartSLOduration=2.009047595 podStartE2EDuration="6.704255662s" totalImagesPullingTime="4.695208067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:39.702210483 +0000 UTC m=+594.064162971" watchObservedRunningTime="2026-09-14 04:56:39.704255662 +0000 UTC m=+594.066208105"
Sep 14 04:56:42.637614 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:42.637577    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:43.643404 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:43.643369    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-q2fl6"
Sep 14 04:56:44.609886 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.609856    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"]
Sep 14 04:56:44.613713 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.613682    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.626266 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.626238    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"]
Sep 14 04:56:44.741562 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.741523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kqmc\" (UniqueName: \"kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.742002 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.741576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.842901 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.842850    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.843145 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.842995    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kqmc\" (UniqueName: \"kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.843285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.843263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.851721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.851683    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kqmc\" (UniqueName: \"kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.924450 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.924330    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:44.956011 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.955974    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"]
Sep 14 04:56:44.956239 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.956212    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" containerName="manager" containerID="cri-o://c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb" gracePeriod=2
Sep 14 04:56:44.966750 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.966651    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"]
Sep 14 04:56:44.979415 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.978852    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"]
Sep 14 04:56:44.987168 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.987134    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"]
Sep 14 04:56:44.993621 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.993588    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"]
Sep 14 04:56:44.994376 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.994336    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" containerName="manager" containerID="cri-o://f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9" gracePeriod=2
Sep 14 04:56:44.996845 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:44.996816    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:44.997551    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:44.999769    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.000001    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.000814    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" containerName="manager"
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.000834    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" containerName="manager"
Sep 14 04:56:45.001542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.001077    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" containerName="manager"
Sep 14 04:56:45.003427 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.003371    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.006005 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.005907    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.008843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.008813    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"]
Sep 14 04:56:45.009785 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.008997    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.013314 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.013286    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.025679 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.025633    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:45.029339 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.029314    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:56:45.030038 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.030014    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" containerName="manager"
Sep 14 04:56:45.030150 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.030140    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" containerName="manager"
Sep 14 04:56:45.030367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.030356    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" containerName="manager"
Sep 14 04:56:45.034251 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.034228    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"]
Sep 14 04:56:45.034452 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.034390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.038580 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.038556    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:45.048777 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.048724    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.050041 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.049510    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:56:45.051611 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.051578    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.053552 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.053516    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.054432 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.054406    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"]
Sep 14 04:56:45.146431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.146153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.146431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.146220    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt9qg\" (UniqueName: \"kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.146431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.146304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk2dh\" (UniqueName: \"kubernetes.io/projected/44961df4-cacd-4bac-acf8-ad26323b4c18-kube-api-access-rk2dh\") pod \"limitador-operator-controller-manager-74495c69f8-8nwc5\" (UID: \"44961df4-cacd-4bac-acf8-ad26323b4c18\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:45.146431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.146331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.146431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.146366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7s4\" (UniqueName: \"kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.181453 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.181376    2571 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 14 04:56:45.181453 ip-10-0-137-105 kubenswrapper[2571]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-x4bmz_kuadrant-system_34c0b29f-c859-4483-8e87-1e5c31b1296c_0(42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-x4bmz to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172" Netns:"/var/run/netns/91066bb8-8de3-46c7-a297-64abd9b9c8b3" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-x4bmz;K8S_POD_INFRA_CONTAINER_ID=42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172;K8S_POD_UID=34c0b29f-c859-4483-8e87-1e5c31b1296c" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz/34c0b29f-c859-4483-8e87-1e5c31b1296c:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz 42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:24 [10.134.0.36/23]
Sep 14 04:56:45.181453 ip-10-0-137-105 kubenswrapper[2571]:         '
Sep 14 04:56:45.181453 ip-10-0-137-105 kubenswrapper[2571]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 14 04:56:45.181453 ip-10-0-137-105 kubenswrapper[2571]:  >
Sep 14 04:56:45.181810 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.181474    2571 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 14 04:56:45.181810 ip-10-0-137-105 kubenswrapper[2571]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-x4bmz_kuadrant-system_34c0b29f-c859-4483-8e87-1e5c31b1296c_0(42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-x4bmz to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172" Netns:"/var/run/netns/91066bb8-8de3-46c7-a297-64abd9b9c8b3" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-x4bmz;K8S_POD_INFRA_CONTAINER_ID=42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172;K8S_POD_UID=34c0b29f-c859-4483-8e87-1e5c31b1296c" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz/34c0b29f-c859-4483-8e87-1e5c31b1296c:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz 42afa1f9a4f4ef363cada12cd4c3cb5335b767be64514b1bac515a8f9ee41172 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:24 [10.134.0.36/23]
Sep 14 04:56:45.181810 ip-10-0-137-105 kubenswrapper[2571]:         '
Sep 14 04:56:45.181810 ip-10-0-137-105 kubenswrapper[2571]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 14 04:56:45.181810 ip-10-0-137-105 kubenswrapper[2571]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.241131 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.241107    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.243975 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.243921    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.245661 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.245642    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.246099 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.246076    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.247463 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247442    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk2dh\" (UniqueName: \"kubernetes.io/projected/44961df4-cacd-4bac-acf8-ad26323b4c18-kube-api-access-rk2dh\") pod \"limitador-operator-controller-manager-74495c69f8-8nwc5\" (UID: \"44961df4-cacd-4bac-acf8-ad26323b4c18\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:45.247520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247475    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.247520 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7s4\" (UniqueName: \"kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.247599 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247552    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.247599 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt9qg\" (UniqueName: \"kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.247851 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.247973 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.247884    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.248175 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.248046    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.251907 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.251883    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.258103 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.258073    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk2dh\" (UniqueName: \"kubernetes.io/projected/44961df4-cacd-4bac-acf8-ad26323b4c18-kube-api-access-rk2dh\") pod \"limitador-operator-controller-manager-74495c69f8-8nwc5\" (UID: \"44961df4-cacd-4bac-acf8-ad26323b4c18\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:45.258222 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.258073    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt9qg\" (UniqueName: \"kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-crngn\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.260049 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.260019    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7s4\" (UniqueName: \"kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-k5bbc\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.348381 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.348341    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d4w6\" (UniqueName: \"kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6\") pod \"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4\" (UID: \"d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4\") "
Sep 14 04:56:45.348554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.348393    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume\") pod \"0c4cae6e-e188-40ff-9052-d5398c048575\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") "
Sep 14 04:56:45.348554 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.348415    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk6xj\" (UniqueName: \"kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj\") pod \"0c4cae6e-e188-40ff-9052-d5398c048575\" (UID: \"0c4cae6e-e188-40ff-9052-d5398c048575\") "
Sep 14 04:56:45.348999 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.348923    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0c4cae6e-e188-40ff-9052-d5398c048575" (UID: "0c4cae6e-e188-40ff-9052-d5398c048575"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:56:45.350892 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.350865    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj" (OuterVolumeSpecName: "kube-api-access-rk6xj") pod "0c4cae6e-e188-40ff-9052-d5398c048575" (UID: "0c4cae6e-e188-40ff-9052-d5398c048575"). InnerVolumeSpecName "kube-api-access-rk6xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:56:45.351044 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.350903    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6" (OuterVolumeSpecName: "kube-api-access-2d4w6") pod "d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" (UID: "d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4"). InnerVolumeSpecName "kube-api-access-2d4w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:56:45.390268 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.390235    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:45.397834 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.397806    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:45.405117 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.405084    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:45.449790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.449749    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d4w6\" (UniqueName: \"kubernetes.io/projected/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4-kube-api-access-2d4w6\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:45.449790 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.449791    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c4cae6e-e188-40ff-9052-d5398c048575-extensions-socket-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:45.450035 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.449808    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk6xj\" (UniqueName: \"kubernetes.io/projected/0c4cae6e-e188-40ff-9052-d5398c048575-kube-api-access-rk6xj\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:45.677367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.677344    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:45.680272 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.680241    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:56:45.680783 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:45.680753    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2740a3fd_4932_46cc_961c_299ebf83adc7.slice/crio-d769ee4835e15f1ad27ba971fcab6451f268ba68ad504a042a1ed47eb87bd8d8 WatchSource:0}: Error finding container d769ee4835e15f1ad27ba971fcab6451f268ba68ad504a042a1ed47eb87bd8d8: Status 404 returned error can't find the container with id d769ee4835e15f1ad27ba971fcab6451f268ba68ad504a042a1ed47eb87bd8d8
Sep 14 04:56:45.684012 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:45.683985    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade7f0fb_3290_4ecb_80a5_12e34470f207.slice/crio-49296cf42b051b3cf1e88db8b7ec2c812eb1a7f4265892a75e0f6e3325364f46 WatchSource:0}: Error finding container 49296cf42b051b3cf1e88db8b7ec2c812eb1a7f4265892a75e0f6e3325364f46: Status 404 returned error can't find the container with id 49296cf42b051b3cf1e88db8b7ec2c812eb1a7f4265892a75e0f6e3325364f46
Sep 14 04:56:45.692675 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.692653    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"]
Sep 14 04:56:45.695587 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:45.695561    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44961df4_cacd_4bac_acf8_ad26323b4c18.slice/crio-a5cf4fb2d83ebcd032ddb4e9e8772e52021fc25508ed5d28ff760663a277a32d WatchSource:0}: Error finding container a5cf4fb2d83ebcd032ddb4e9e8772e52021fc25508ed5d28ff760663a277a32d: Status 404 returned error can't find the container with id a5cf4fb2d83ebcd032ddb4e9e8772e52021fc25508ed5d28ff760663a277a32d
Sep 14 04:56:45.715891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.715861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" event={"ID":"ade7f0fb-3290-4ecb-80a5-12e34470f207","Type":"ContainerStarted","Data":"49296cf42b051b3cf1e88db8b7ec2c812eb1a7f4265892a75e0f6e3325364f46"}
Sep 14 04:56:45.717579 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.717294    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5" event={"ID":"44961df4-cacd-4bac-acf8-ad26323b4c18","Type":"ContainerStarted","Data":"a5cf4fb2d83ebcd032ddb4e9e8772e52021fc25508ed5d28ff760663a277a32d"}
Sep 14 04:56:45.718994 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.718971    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" event={"ID":"2740a3fd-4932-46cc-961c-299ebf83adc7","Type":"ContainerStarted","Data":"d769ee4835e15f1ad27ba971fcab6451f268ba68ad504a042a1ed47eb87bd8d8"}
Sep 14 04:56:45.720923 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.720894    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c4cae6e-e188-40ff-9052-d5398c048575" containerID="c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb" exitCode=0
Sep 14 04:56:45.721052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.720971    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.721052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.720987    2571 scope.go:117] "RemoveContainer" containerID="c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb"
Sep 14 04:56:45.723356 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.723331    2571 generic.go:356] "Generic (PLEG): container finished" podID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" containerID="f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9" exitCode=0
Sep 14 04:56:45.723456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.723404    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.723456 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.723426    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.723855 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.723726    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.726263 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.726229    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.728370 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.728339    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.730359 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.730334    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.732228 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.732202    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.785987 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.785898    2571 scope.go:117] "RemoveContainer" containerID="c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb"
Sep 14 04:56:45.786334 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.786315    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb\": container with ID starting with c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb not found: ID does not exist" containerID="c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb"
Sep 14 04:56:45.786378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.786343    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb"} err="failed to get container status \"c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb\": rpc error: code = NotFound desc = could not find container \"c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb\": container with ID starting with c8e08807771b2d700a1ca291a269c724bd76dc3b404be01d060fb5fcbbb666fb not found: ID does not exist"
Sep 14 04:56:45.786378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.786362    2571 scope.go:117] "RemoveContainer" containerID="f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9"
Sep 14 04:56:45.789284 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.789265    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.791545 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.791518    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.793584 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.793560    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.795531 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.795509    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.813769 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.813626    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.816263 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.816231    2571 scope.go:117] "RemoveContainer" containerID="f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9"
Sep 14 04:56:45.816713 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.816691    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9\": container with ID starting with f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9 not found: ID does not exist" containerID="f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9"
Sep 14 04:56:45.816800 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.816727    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9"} err="failed to get container status \"f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9\": rpc error: code = NotFound desc = could not find container \"f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9\": container with ID starting with f3a4e8225b2ccf096e93b42e977883faf175196e6377f92285607800126a0ed9 not found: ID does not exist"
Sep 14 04:56:45.817291 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.817260    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.819659 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.819625    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.821964 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.821914    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:45.827003 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.826964    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:45.829225 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:45.829195    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:45.955037 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.954992    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kqmc\" (UniqueName: \"kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc\") pod \"34c0b29f-c859-4483-8e87-1e5c31b1296c\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") "
Sep 14 04:56:45.955184 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.955054    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume\") pod \"34c0b29f-c859-4483-8e87-1e5c31b1296c\" (UID: \"34c0b29f-c859-4483-8e87-1e5c31b1296c\") "
Sep 14 04:56:45.955343 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.955317    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c0b29f-c859-4483-8e87-1e5c31b1296c" (UID: "34c0b29f-c859-4483-8e87-1e5c31b1296c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:56:45.957399 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:45.957377    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc" (OuterVolumeSpecName: "kube-api-access-2kqmc") pod "34c0b29f-c859-4483-8e87-1e5c31b1296c" (UID: "34c0b29f-c859-4483-8e87-1e5c31b1296c"). InnerVolumeSpecName "kube-api-access-2kqmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:56:46.055892 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.055856    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kqmc\" (UniqueName: \"kubernetes.io/projected/34c0b29f-c859-4483-8e87-1e5c31b1296c-kube-api-access-2kqmc\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:46.055892 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.055890    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c0b29f-c859-4483-8e87-1e5c31b1296c-extensions-socket-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:46.101286 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.101260    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:56:46.101801 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.101779    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 04:56:46.106672 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.106651    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:56:46.109597 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.109567    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 04:56:46.193165 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:46.193131    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:46.195448 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:46.195424    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-55xrj\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-55xrj"
Sep 14 04:56:46.195505 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.195444    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" path="/var/lib/kubelet/pods/0c4cae6e-e188-40ff-9052-d5398c048575/volumes"
Sep 14 04:56:46.195820 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.195806    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" path="/var/lib/kubelet/pods/34c0b29f-c859-4483-8e87-1e5c31b1296c/volumes"
Sep 14 04:56:46.196027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.196015    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4" path="/var/lib/kubelet/pods/d12e6ebe-2ac3-409f-9b8d-71daa0ffd6e4/volumes"
Sep 14 04:56:46.197702 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:46.197681    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-wx88h\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0c4cae6e-e188-40ff-9052-d5398c048575" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wx88h"
Sep 14 04:56:46.701727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.701697    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:46.730033 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.729992    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" event={"ID":"2740a3fd-4932-46cc-961c-299ebf83adc7","Type":"ContainerStarted","Data":"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"}
Sep 14 04:56:46.730217 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.730159    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:46.731820 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.731796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" event={"ID":"ade7f0fb-3290-4ecb-80a5-12e34470f207","Type":"ContainerStarted","Data":"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"}
Sep 14 04:56:46.731960 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.731845    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:46.733577 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.733549    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5" event={"ID":"44961df4-cacd-4bac-acf8-ad26323b4c18","Type":"ContainerStarted","Data":"1e3f5adef45ea0f1b7b5633cec2cd2af3e5af5dbc7fb69292e1a14fd7d959344"}
Sep 14 04:56:46.733691 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.733589    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:46.733691 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.733603    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:46.755337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.755273    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" podStartSLOduration=2.755250748 podStartE2EDuration="2.755250748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:46.753875966 +0000 UTC m=+601.115828410" watchObservedRunningTime="2026-09-14 04:56:46.755250748 +0000 UTC m=+601.117203190"
Sep 14 04:56:46.779186 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.779131    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" podStartSLOduration=2.779112006 podStartE2EDuration="2.779112006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:46.777453786 +0000 UTC m=+601.139406229" watchObservedRunningTime="2026-09-14 04:56:46.779112006 +0000 UTC m=+601.141064451"
Sep 14 04:56:46.779863 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:46.779831    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-x4bmz\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="34c0b29f-c859-4483-8e87-1e5c31b1296c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-x4bmz"
Sep 14 04:56:46.784392 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.784366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:46.802048 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:46.801994    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5" podStartSLOduration=2.801974221 podStartE2EDuration="2.801974221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:46.800755943 +0000 UTC m=+601.162708387" watchObservedRunningTime="2026-09-14 04:56:46.801974221 +0000 UTC m=+601.163926664"
Sep 14 04:56:49.989042 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:49.988986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:50.070729 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:50.070699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:53.907415 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:53.907327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:53.965119 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:53.965059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:54.528356 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:54.528321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:54.629184 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:54.629153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:57.740615 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:57.740579    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:57.741123 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:57.740672    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-8nwc5"
Sep 14 04:56:57.741123 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:57.740918    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:56:57.817589 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:57.817549    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:57.817791 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:57.817749    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" podUID="2740a3fd-4932-46cc-961c-299ebf83adc7" containerName="manager" containerID="cri-o://725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0" gracePeriod=10
Sep 14 04:56:58.062247 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.062212    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 04:56:58.066480 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.066452    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.079516 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.079482    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:58.081972 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.081908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 04:56:58.163697 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.163666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq7s4\" (UniqueName: \"kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4\") pod \"2740a3fd-4932-46cc-961c-299ebf83adc7\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") "
Sep 14 04:56:58.163884 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.163716    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume\") pod \"2740a3fd-4932-46cc-961c-299ebf83adc7\" (UID: \"2740a3fd-4932-46cc-961c-299ebf83adc7\") "
Sep 14 04:56:58.163884 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.163809    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.163999 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.163894    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgj8\" (UniqueName: \"kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.164180 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.164159    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2740a3fd-4932-46cc-961c-299ebf83adc7" (UID: "2740a3fd-4932-46cc-961c-299ebf83adc7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:56:58.166177 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.166147    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4" (OuterVolumeSpecName: "kube-api-access-vq7s4") pod "2740a3fd-4932-46cc-961c-299ebf83adc7" (UID: "2740a3fd-4932-46cc-961c-299ebf83adc7"). InnerVolumeSpecName "kube-api-access-vq7s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:56:58.264594 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.264550    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.264778 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.264643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgj8\" (UniqueName: \"kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.264778 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.264718    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq7s4\" (UniqueName: \"kubernetes.io/projected/2740a3fd-4932-46cc-961c-299ebf83adc7-kube-api-access-vq7s4\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:58.264778 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.264729    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2740a3fd-4932-46cc-961c-299ebf83adc7-extensions-socket-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:56:58.265051 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.265029    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.276335 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.276251    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgj8\" (UniqueName: \"kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8\") pod \"kuadrant-operator-controller-manager-5769f99d9f-b92qp\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.386242 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.386193    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.676089 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.675944    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 04:56:58.749873 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:56:58.680114    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5de6f0_d879_46bc_9efc_391fefa2ded3.slice/crio-385f5f1c46cee6eb8ca80e8ec0a4beb3345057f4efd3fcf1c9c768db82deae53 WatchSource:0}: Error finding container 385f5f1c46cee6eb8ca80e8ec0a4beb3345057f4efd3fcf1c9c768db82deae53: Status 404 returned error can't find the container with id 385f5f1c46cee6eb8ca80e8ec0a4beb3345057f4efd3fcf1c9c768db82deae53
Sep 14 04:56:58.795226 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.795185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" event={"ID":"6a5de6f0-d879-46bc-9efc-391fefa2ded3","Type":"ContainerStarted","Data":"1a5dd21851a2ae9d9a7e5803ca500b915b12718963df715e618f033594ebb215"}
Sep 14 04:56:58.795421 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.795248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" event={"ID":"6a5de6f0-d879-46bc-9efc-391fefa2ded3","Type":"ContainerStarted","Data":"385f5f1c46cee6eb8ca80e8ec0a4beb3345057f4efd3fcf1c9c768db82deae53"}
Sep 14 04:56:58.795421 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.795275    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:56:58.796721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.796690    2571 generic.go:356] "Generic (PLEG): container finished" podID="2740a3fd-4932-46cc-961c-299ebf83adc7" containerID="725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0" exitCode=0
Sep 14 04:56:58.796891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.796744    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" event={"ID":"2740a3fd-4932-46cc-961c-299ebf83adc7","Type":"ContainerDied","Data":"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"}
Sep 14 04:56:58.796891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.796785    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc" event={"ID":"2740a3fd-4932-46cc-961c-299ebf83adc7","Type":"ContainerDied","Data":"d769ee4835e15f1ad27ba971fcab6451f268ba68ad504a042a1ed47eb87bd8d8"}
Sep 14 04:56:58.796891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.796814    2571 scope.go:117] "RemoveContainer" containerID="725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"
Sep 14 04:56:58.796891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.796746    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"
Sep 14 04:56:58.805728 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.805704    2571 scope.go:117] "RemoveContainer" containerID="725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"
Sep 14 04:56:58.806196 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:56:58.806171    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0\": container with ID starting with 725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0 not found: ID does not exist" containerID="725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"
Sep 14 04:56:58.806286 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.806203    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0"} err="failed to get container status \"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0\": rpc error: code = NotFound desc = could not find container \"725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0\": container with ID starting with 725486b777ac9ba96ba25620188953f980721fb7852f2d5465359e20d60ad3d0 not found: ID does not exist"
Sep 14 04:56:58.819817 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.819768    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" podStartSLOduration=0.819752986 podStartE2EDuration="819.752986ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:56:58.818750753 +0000 UTC m=+613.180703196" watchObservedRunningTime="2026-09-14 04:56:58.819752986 +0000 UTC m=+613.181705429"
Sep 14 04:56:58.849214 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.849177    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:58.859373 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.859334    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-k5bbc"]
Sep 14 04:56:58.941297 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:58.941219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:59.044205 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:59.044168    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:59.306674 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:59.306642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:59.408717 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:59.408677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:59.723990 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:59.723955    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:56:59.828233 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:56:59.828196    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:00.195205 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:00.195123    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2740a3fd-4932-46cc-961c-299ebf83adc7" path="/var/lib/kubelet/pods/2740a3fd-4932-46cc-961c-299ebf83adc7/volumes"
Sep 14 04:57:05.200953 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:05.200894    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:05.264654 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:05.264622    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:09.807400 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:09.807309    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 04:57:09.912337 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:09.912304    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:57:09.912563 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:09.912540    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" podUID="ade7f0fb-3290-4ecb-80a5-12e34470f207" containerName="manager" containerID="cri-o://44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46" gracePeriod=10
Sep 14 04:57:09.983162 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:09.983124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:10.078290 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.078198    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:10.165212 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.165183    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:57:10.272541 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.272505    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt9qg\" (UniqueName: \"kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg\") pod \"ade7f0fb-3290-4ecb-80a5-12e34470f207\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") "
Sep 14 04:57:10.272736 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.272593    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume\") pod \"ade7f0fb-3290-4ecb-80a5-12e34470f207\" (UID: \"ade7f0fb-3290-4ecb-80a5-12e34470f207\") "
Sep 14 04:57:10.272989 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.272926    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ade7f0fb-3290-4ecb-80a5-12e34470f207" (UID: "ade7f0fb-3290-4ecb-80a5-12e34470f207"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 04:57:10.274775 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.274748    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg" (OuterVolumeSpecName: "kube-api-access-kt9qg") pod "ade7f0fb-3290-4ecb-80a5-12e34470f207" (UID: "ade7f0fb-3290-4ecb-80a5-12e34470f207"). InnerVolumeSpecName "kube-api-access-kt9qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:57:10.373391 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.373286    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt9qg\" (UniqueName: \"kubernetes.io/projected/ade7f0fb-3290-4ecb-80a5-12e34470f207-kube-api-access-kt9qg\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:57:10.373391 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.373320    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ade7f0fb-3290-4ecb-80a5-12e34470f207-extensions-socket-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:57:10.867285 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.867248    2571 generic.go:356] "Generic (PLEG): container finished" podID="ade7f0fb-3290-4ecb-80a5-12e34470f207" containerID="44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46" exitCode=0
Sep 14 04:57:10.867738 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.867298    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"
Sep 14 04:57:10.867738 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.867335    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" event={"ID":"ade7f0fb-3290-4ecb-80a5-12e34470f207","Type":"ContainerDied","Data":"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"}
Sep 14 04:57:10.867738 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.867386    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn" event={"ID":"ade7f0fb-3290-4ecb-80a5-12e34470f207","Type":"ContainerDied","Data":"49296cf42b051b3cf1e88db8b7ec2c812eb1a7f4265892a75e0f6e3325364f46"}
Sep 14 04:57:10.867738 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.867410    2571 scope.go:117] "RemoveContainer" containerID="44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"
Sep 14 04:57:10.877027 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.877003    2571 scope.go:117] "RemoveContainer" containerID="44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"
Sep 14 04:57:10.877350 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:57:10.877328    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46\": container with ID starting with 44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46 not found: ID does not exist" containerID="44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"
Sep 14 04:57:10.877396 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.877361    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46"} err="failed to get container status \"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46\": rpc error: code = NotFound desc = could not find container \"44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46\": container with ID starting with 44f54585765e923d8ae172b99137b173861d3b7d5e098eec8bfd4cf31778ae46 not found: ID does not exist"
Sep 14 04:57:10.902869 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.902834    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:57:10.910498 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:10.910461    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-crngn"]
Sep 14 04:57:12.194569 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:12.194534    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ade7f0fb-3290-4ecb-80a5-12e34470f207" path="/var/lib/kubelet/pods/ade7f0fb-3290-4ecb-80a5-12e34470f207/volumes"
Sep 14 04:57:14.225295 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:14.225254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:14.426740 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:14.426709    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:26.767602 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:26.767533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:26.928575 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:26.928523    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:28.668363 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.668327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:28.671803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.671773    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:28.672176 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672161    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2740a3fd-4932-46cc-961c-299ebf83adc7" containerName="manager"
Sep 14 04:57:28.672234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672177    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2740a3fd-4932-46cc-961c-299ebf83adc7" containerName="manager"
Sep 14 04:57:28.672234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672189    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade7f0fb-3290-4ecb-80a5-12e34470f207" containerName="manager"
Sep 14 04:57:28.672234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672195    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade7f0fb-3290-4ecb-80a5-12e34470f207" containerName="manager"
Sep 14 04:57:28.672332 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672255    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade7f0fb-3290-4ecb-80a5-12e34470f207" containerName="manager"
Sep 14 04:57:28.672332 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.672265    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2740a3fd-4932-46cc-961c-299ebf83adc7" containerName="manager"
Sep 14 04:57:28.676549 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.676528    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.679431 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.679401    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 04:57:28.684557 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.684532    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:28.762732 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.762691    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:28.775426 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.775384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:28.832575 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.832544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:28.845710 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.845673    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7sfp\" (UniqueName: \"kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.845824 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.845798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.947188 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.947089    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.947188 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.947147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sfp\" (UniqueName: \"kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.947731 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.947705    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.957276 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.957241    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sfp\" (UniqueName: \"kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp\") pod \"limitador-limitador-8588749977-kbsw6\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") " pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:28.987699 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:28.987656    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:29.265443 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.265420    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:29.267832 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:57:29.267798    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda949e7bf_4052_4356_bd3d_6ad68c36c5ee.slice/crio-e1a297efa63b2d780887649918021640821ff5cbe08d7220dbe7fdb9d55aa359 WatchSource:0}: Error finding container e1a297efa63b2d780887649918021640821ff5cbe08d7220dbe7fdb9d55aa359: Status 404 returned error can't find the container with id e1a297efa63b2d780887649918021640821ff5cbe08d7220dbe7fdb9d55aa359
Sep 14 04:57:29.461891 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.461854    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:29.466545 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.466519    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:29.469639 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.469613    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pkggr\""
Sep 14 04:57:29.474247 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.474220    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:29.494980 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.494948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:29.552874 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.552770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzq42\" (UniqueName: \"kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42\") pod \"authorino-7bcfd8f9cd-v8gpw\" (UID: \"a30843e4-b13e-41be-b04d-806d460ca19d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:29.598578 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.598530    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:29.653795 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.653753    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzq42\" (UniqueName: \"kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42\") pod \"authorino-7bcfd8f9cd-v8gpw\" (UID: \"a30843e4-b13e-41be-b04d-806d460ca19d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:29.663198 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.663160    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzq42\" (UniqueName: \"kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42\") pod \"authorino-7bcfd8f9cd-v8gpw\" (UID: \"a30843e4-b13e-41be-b04d-806d460ca19d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:29.778367 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.778329    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:29.964914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:29.964872    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" event={"ID":"a949e7bf-4052-4356-bd3d-6ad68c36c5ee","Type":"ContainerStarted","Data":"e1a297efa63b2d780887649918021640821ff5cbe08d7220dbe7fdb9d55aa359"}
Sep 14 04:57:30.058652 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:30.056021    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:30.062705 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:57:30.062668    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda30843e4_b13e_41be_b04d_806d460ca19d.slice/crio-93f023018f4afba773f862acdc5528388f8260d15d84a988c4fe34cc195eff32 WatchSource:0}: Error finding container 93f023018f4afba773f862acdc5528388f8260d15d84a988c4fe34cc195eff32: Status 404 returned error can't find the container with id 93f023018f4afba773f862acdc5528388f8260d15d84a988c4fe34cc195eff32
Sep 14 04:57:30.971826 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:30.971784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" event={"ID":"a949e7bf-4052-4356-bd3d-6ad68c36c5ee","Type":"ContainerStarted","Data":"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"}
Sep 14 04:57:30.972296 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:30.971969    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:30.973150 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:30.973125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" event={"ID":"a30843e4-b13e-41be-b04d-806d460ca19d","Type":"ContainerStarted","Data":"93f023018f4afba773f862acdc5528388f8260d15d84a988c4fe34cc195eff32"}
Sep 14 04:57:30.991830 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:30.991781    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" podStartSLOduration=1.8040464410000001 podStartE2EDuration="2.991761535s" totalImagesPullingTime="1.187715094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:57:30.990500407 +0000 UTC m=+645.352452844" watchObservedRunningTime="2026-09-14 04:57:30.991761535 +0000 UTC m=+645.353713978"
Sep 14 04:57:33.976060 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:33.976023    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:37.013707 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.013661    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" event={"ID":"a30843e4-b13e-41be-b04d-806d460ca19d","Type":"ContainerStarted","Data":"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"}
Sep 14 04:57:37.014179 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.013723    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" podUID="a30843e4-b13e-41be-b04d-806d460ca19d" containerName="authorino" containerID="cri-o://fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401" gracePeriod=30
Sep 14 04:57:37.029716 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.029662    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" podStartSLOduration=2.126282025 podStartE2EDuration="8.029643898s" totalImagesPullingTime="5.903361873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:57:37.02941214 +0000 UTC m=+651.391364584" watchObservedRunningTime="2026-09-14 04:57:37.029643898 +0000 UTC m=+651.391596341"
Sep 14 04:57:37.266235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.266165    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:37.424184 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.424142    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzq42\" (UniqueName: \"kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42\") pod \"a30843e4-b13e-41be-b04d-806d460ca19d\" (UID: \"a30843e4-b13e-41be-b04d-806d460ca19d\") "
Sep 14 04:57:37.426339 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.426314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42" (OuterVolumeSpecName: "kube-api-access-xzq42") pod "a30843e4-b13e-41be-b04d-806d460ca19d" (UID: "a30843e4-b13e-41be-b04d-806d460ca19d"). InnerVolumeSpecName "kube-api-access-xzq42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:57:37.525063 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:37.524964    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzq42\" (UniqueName: \"kubernetes.io/projected/a30843e4-b13e-41be-b04d-806d460ca19d-kube-api-access-xzq42\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:57:38.020452 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.020415    2571 generic.go:356] "Generic (PLEG): container finished" podID="a30843e4-b13e-41be-b04d-806d460ca19d" containerID="fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401" exitCode=0
Sep 14 04:57:38.020872 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.020461    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw"
Sep 14 04:57:38.020872 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.020485    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" event={"ID":"a30843e4-b13e-41be-b04d-806d460ca19d","Type":"ContainerDied","Data":"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"}
Sep 14 04:57:38.020872 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.020519    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-v8gpw" event={"ID":"a30843e4-b13e-41be-b04d-806d460ca19d","Type":"ContainerDied","Data":"93f023018f4afba773f862acdc5528388f8260d15d84a988c4fe34cc195eff32"}
Sep 14 04:57:38.020872 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.020535    2571 scope.go:117] "RemoveContainer" containerID="fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"
Sep 14 04:57:38.029721 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.029703    2571 scope.go:117] "RemoveContainer" containerID="fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"
Sep 14 04:57:38.030146 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:57:38.030123    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401\": container with ID starting with fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401 not found: ID does not exist" containerID="fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"
Sep 14 04:57:38.030223 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.030155    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401"} err="failed to get container status \"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401\": rpc error: code = NotFound desc = could not find container \"fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401\": container with ID starting with fdcfd4976a872850f3d841c21a8059b65427316cdbb5907b8eb02831f351a401 not found: ID does not exist"
Sep 14 04:57:38.042383 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.042354    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:38.046340 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.046314    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-v8gpw"]
Sep 14 04:57:38.194054 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:38.194012    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a30843e4-b13e-41be-b04d-806d460ca19d" path="/var/lib/kubelet/pods/a30843e4-b13e-41be-b04d-806d460ca19d/volumes"
Sep 14 04:57:41.979814 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:41.979776    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:42.195354 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:42.195302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:42.295954 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:42.295859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:42.981347 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:42.981315    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:42.981821 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:42.981509    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" podUID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" containerName="limitador" containerID="cri-o://d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af" gracePeriod=30
Sep 14 04:57:43.535186 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.535149    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:43.573092 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.572988    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file\") pod \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") "
Sep 14 04:57:43.573235 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.573109    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7sfp\" (UniqueName: \"kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp\") pod \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\" (UID: \"a949e7bf-4052-4356-bd3d-6ad68c36c5ee\") "
Sep 14 04:57:43.573502 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.573469    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file" (OuterVolumeSpecName: "config-file") pod "a949e7bf-4052-4356-bd3d-6ad68c36c5ee" (UID: "a949e7bf-4052-4356-bd3d-6ad68c36c5ee"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 04:57:43.575403 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.575374    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp" (OuterVolumeSpecName: "kube-api-access-w7sfp") pod "a949e7bf-4052-4356-bd3d-6ad68c36c5ee" (UID: "a949e7bf-4052-4356-bd3d-6ad68c36c5ee"). InnerVolumeSpecName "kube-api-access-w7sfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:57:43.674088 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.674043    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7sfp\" (UniqueName: \"kubernetes.io/projected/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-kube-api-access-w7sfp\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:57:43.674088 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:43.674078    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a949e7bf-4052-4356-bd3d-6ad68c36c5ee-config-file\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:57:44.052459 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.052420    2571 generic.go:356] "Generic (PLEG): container finished" podID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" containerID="d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af" exitCode=0
Sep 14 04:57:44.052852 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.052466    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6"
Sep 14 04:57:44.052852 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.052468    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" event={"ID":"a949e7bf-4052-4356-bd3d-6ad68c36c5ee","Type":"ContainerDied","Data":"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"}
Sep 14 04:57:44.052852 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.052523    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-kbsw6" event={"ID":"a949e7bf-4052-4356-bd3d-6ad68c36c5ee","Type":"ContainerDied","Data":"e1a297efa63b2d780887649918021640821ff5cbe08d7220dbe7fdb9d55aa359"}
Sep 14 04:57:44.052852 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.052558    2571 scope.go:117] "RemoveContainer" containerID="d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"
Sep 14 04:57:44.061532 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.061512    2571 scope.go:117] "RemoveContainer" containerID="d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"
Sep 14 04:57:44.061855 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:57:44.061836    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af\": container with ID starting with d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af not found: ID does not exist" containerID="d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"
Sep 14 04:57:44.061908 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.061868    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af"} err="failed to get container status \"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af\": rpc error: code = NotFound desc = could not find container \"d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af\": container with ID starting with d6a0504b9ea631b5086496b6e1d16dde597022c24c6d8db6dfeed0ba5fe707af not found: ID does not exist"
Sep 14 04:57:44.076065 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.076034    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:44.079970 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.079944    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-kbsw6"]
Sep 14 04:57:44.194217 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:44.194180    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" path="/var/lib/kubelet/pods/a949e7bf-4052-4356-bd3d-6ad68c36c5ee/volumes"
Sep 14 04:57:47.417359 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:47.417325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:47.997260 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:47.997218    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:48.087085 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.087053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:48.501238 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501204    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"]
Sep 14 04:57:48.501617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501579    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" containerName="limitador"
Sep 14 04:57:48.501617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501589    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" containerName="limitador"
Sep 14 04:57:48.501617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501608    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a30843e4-b13e-41be-b04d-806d460ca19d" containerName="authorino"
Sep 14 04:57:48.501617 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501613    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a30843e4-b13e-41be-b04d-806d460ca19d" containerName="authorino"
Sep 14 04:57:48.501831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501663    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a949e7bf-4052-4356-bd3d-6ad68c36c5ee" containerName="limitador"
Sep 14 04:57:48.501831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.501672    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a30843e4-b13e-41be-b04d-806d460ca19d" containerName="authorino"
Sep 14 04:57:48.506422 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.506397    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.508900 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.508859    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 04:57:48.509084 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.508924    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 04:57:48.509811 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.509793    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-klv7b\""
Sep 14 04:57:48.509947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.509818    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 14 04:57:48.512703 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.512679    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"]
Sep 14 04:57:48.529844 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.529802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:48.616783 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.616742    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzh6x\" (UniqueName: \"kubernetes.io/projected/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-kube-api-access-lzh6x\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.616989 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.616813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-data\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.718185 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.718147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-data\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.718321 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.718230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzh6x\" (UniqueName: \"kubernetes.io/projected/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-kube-api-access-lzh6x\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.718626 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.718602    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-data\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.726412 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.726380    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzh6x\" (UniqueName: \"kubernetes.io/projected/7beb41a7-9ae2-4fca-9d3a-d69a247cb727-kube-api-access-lzh6x\") pod \"postgres-69fb4594cf-rzcdw\" (UID: \"7beb41a7-9ae2-4fca-9d3a-d69a247cb727\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:48.818356 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:48.818249    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:49.098473 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:49.096075    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"]
Sep 14 04:57:49.101917 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:57:49.101882    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7beb41a7_9ae2_4fca_9d3a_d69a247cb727.slice/crio-c44d091790f666d044ea42575a172b59c769173181f213b6af40dd43b59a7cbc WatchSource:0}: Error finding container c44d091790f666d044ea42575a172b59c769173181f213b6af40dd43b59a7cbc: Status 404 returned error can't find the container with id c44d091790f666d044ea42575a172b59c769173181f213b6af40dd43b59a7cbc
Sep 14 04:57:49.103108 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:49.103084    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 04:57:49.778806 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:49.778769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:50.085062 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:50.084968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw" event={"ID":"7beb41a7-9ae2-4fca-9d3a-d69a247cb727","Type":"ContainerStarted","Data":"c44d091790f666d044ea42575a172b59c769173181f213b6af40dd43b59a7cbc"}
Sep 14 04:57:53.101746 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:53.101710    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw" event={"ID":"7beb41a7-9ae2-4fca-9d3a-d69a247cb727","Type":"ContainerStarted","Data":"75471696dbc84a6c1fad5a8f9d724e19444ecdd36653b5a12ffd17d4e3ced3a9"}
Sep 14 04:57:53.102177 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:53.101815    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:53.122149 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:53.122091    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw" podStartSLOduration=2.06660362 podStartE2EDuration="5.122068028s" totalImagesPullingTime="3.055464408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:57:53.120012101 +0000 UTC m=+667.481964550" watchObservedRunningTime="2026-09-14 04:57:53.122068028 +0000 UTC m=+667.484020471"
Sep 14 04:57:59.137001 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.136968    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-rzcdw"
Sep 14 04:57:59.498053 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.498015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:57:59.754074 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.753986    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:57:59.760238 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.760209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:57:59.764296 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.764270    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pkggr\""
Sep 14 04:57:59.769366 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.769330    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:57:59.823582 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.823547    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmz84\" (UniqueName: \"kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84\") pod \"authorino-557f876f69-nzcgs\" (UID: \"62f61cae-890f-42cb-8b71-bdaccbc67967\") " pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:57:59.924329 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.924289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmz84\" (UniqueName: \"kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84\") pod \"authorino-557f876f69-nzcgs\" (UID: \"62f61cae-890f-42cb-8b71-bdaccbc67967\") " pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:57:59.933299 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.933268    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmz84\" (UniqueName: \"kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84\") pod \"authorino-557f876f69-nzcgs\" (UID: \"62f61cae-890f-42cb-8b71-bdaccbc67967\") " pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:57:59.996161 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.996122    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:57:59.996392 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:57:59.996381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:58:00.025947 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.025892    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:00.032083 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.032055    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:00.041803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.041768    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:00.116158 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.116130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:00.126315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.126281    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwgq\" (UniqueName: \"kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq\") pod \"authorino-544f998b86-4pr4z\" (UID: \"70f710e7-2590-406b-8520-925636a2c307\") " pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:00.227618 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.227581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grwgq\" (UniqueName: \"kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq\") pod \"authorino-544f998b86-4pr4z\" (UID: \"70f710e7-2590-406b-8520-925636a2c307\") " pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:00.237254 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.237227    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwgq\" (UniqueName: \"kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq\") pod \"authorino-544f998b86-4pr4z\" (UID: \"70f710e7-2590-406b-8520-925636a2c307\") " pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:00.276034 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.276006    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:58:00.278405 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:00.278378    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62f61cae_890f_42cb_8b71_bdaccbc67967.slice/crio-7efa2a9e86be3d076c9776be3cec1cbf65efa71925ac393248a034837ac204fc WatchSource:0}: Error finding container 7efa2a9e86be3d076c9776be3cec1cbf65efa71925ac393248a034837ac204fc: Status 404 returned error can't find the container with id 7efa2a9e86be3d076c9776be3cec1cbf65efa71925ac393248a034837ac204fc
Sep 14 04:58:00.345169 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.345128    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:00.621854 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:00.621828    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:00.623500 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:00.623473    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f710e7_2590_406b_8520_925636a2c307.slice/crio-506897f162da81a634d7e3f453343f718f967013b1d5a277cfb98a183fb4b58d WatchSource:0}: Error finding container 506897f162da81a634d7e3f453343f718f967013b1d5a277cfb98a183fb4b58d: Status 404 returned error can't find the container with id 506897f162da81a634d7e3f453343f718f967013b1d5a277cfb98a183fb4b58d
Sep 14 04:58:01.145871 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.145809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-544f998b86-4pr4z" event={"ID":"70f710e7-2590-406b-8520-925636a2c307","Type":"ContainerStarted","Data":"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"}
Sep 14 04:58:01.145871 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.145850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-544f998b86-4pr4z" event={"ID":"70f710e7-2590-406b-8520-925636a2c307","Type":"ContainerStarted","Data":"506897f162da81a634d7e3f453343f718f967013b1d5a277cfb98a183fb4b58d"}
Sep 14 04:58:01.147671 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.147645    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nzcgs" event={"ID":"62f61cae-890f-42cb-8b71-bdaccbc67967","Type":"ContainerStarted","Data":"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"}
Sep 14 04:58:01.147826 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.147679    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nzcgs" event={"ID":"62f61cae-890f-42cb-8b71-bdaccbc67967","Type":"ContainerStarted","Data":"7efa2a9e86be3d076c9776be3cec1cbf65efa71925ac393248a034837ac204fc"}
Sep 14 04:58:01.147826 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.147721    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-nzcgs" podUID="62f61cae-890f-42cb-8b71-bdaccbc67967" containerName="authorino" containerID="cri-o://e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804" gracePeriod=30
Sep 14 04:58:01.164894 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.164834    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-544f998b86-4pr4z" podStartSLOduration=0.763340826 podStartE2EDuration="1.164813755s" totalImagesPullingTime="401.472929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:01.163205668 +0000 UTC m=+675.525158113" watchObservedRunningTime="2026-09-14 04:58:01.164813755 +0000 UTC m=+675.526766203"
Sep 14 04:58:01.184378 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.184312    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-nzcgs" podStartSLOduration=1.765255814 podStartE2EDuration="2.18428678s" totalImagesPullingTime="419.030966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:01.182720539 +0000 UTC m=+675.544672982" watchObservedRunningTime="2026-09-14 04:58:01.18428678 +0000 UTC m=+675.546239224"
Sep 14 04:58:01.413843 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.413810    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:58:01.539715 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.539678    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmz84\" (UniqueName: \"kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84\") pod \"62f61cae-890f-42cb-8b71-bdaccbc67967\" (UID: \"62f61cae-890f-42cb-8b71-bdaccbc67967\") "
Sep 14 04:58:01.541965 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.541939    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84" (OuterVolumeSpecName: "kube-api-access-wmz84") pod "62f61cae-890f-42cb-8b71-bdaccbc67967" (UID: "62f61cae-890f-42cb-8b71-bdaccbc67967"). InnerVolumeSpecName "kube-api-access-wmz84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:58:01.641362 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:01.641272    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmz84\" (UniqueName: \"kubernetes.io/projected/62f61cae-890f-42cb-8b71-bdaccbc67967-kube-api-access-wmz84\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:58:02.153368 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.153330    2571 generic.go:356] "Generic (PLEG): container finished" podID="62f61cae-890f-42cb-8b71-bdaccbc67967" containerID="e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804" exitCode=0
Sep 14 04:58:02.153548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.153391    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nzcgs"
Sep 14 04:58:02.153548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.153422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nzcgs" event={"ID":"62f61cae-890f-42cb-8b71-bdaccbc67967","Type":"ContainerDied","Data":"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"}
Sep 14 04:58:02.153548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.153474    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-nzcgs" event={"ID":"62f61cae-890f-42cb-8b71-bdaccbc67967","Type":"ContainerDied","Data":"7efa2a9e86be3d076c9776be3cec1cbf65efa71925ac393248a034837ac204fc"}
Sep 14 04:58:02.153548 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.153497    2571 scope.go:117] "RemoveContainer" containerID="e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"
Sep 14 04:58:02.162463 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.162446    2571 scope.go:117] "RemoveContainer" containerID="e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"
Sep 14 04:58:02.162753 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:58:02.162733    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804\": container with ID starting with e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804 not found: ID does not exist" containerID="e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"
Sep 14 04:58:02.162803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.162763    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804"} err="failed to get container status \"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804\": rpc error: code = NotFound desc = could not find container \"e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804\": container with ID starting with e3a6c4fa6366e1a7ecee554a99937482e4737126a7ccaa3becd682e8c68a4804 not found: ID does not exist"
Sep 14 04:58:02.175910 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.175883    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:58:02.178696 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.178673    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-nzcgs"]
Sep 14 04:58:02.194464 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:02.194382    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f61cae-890f-42cb-8b71-bdaccbc67967" path="/var/lib/kubelet/pods/62f61cae-890f-42cb-8b71-bdaccbc67967/volumes"
Sep 14 04:58:03.131961 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.131909    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:03.160225 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.160187    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-544f998b86-4pr4z" podUID="70f710e7-2590-406b-8520-925636a2c307" containerName="authorino" containerID="cri-o://2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524" gracePeriod=30
Sep 14 04:58:03.412424 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.412397    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:03.556863 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.556812    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grwgq\" (UniqueName: \"kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq\") pod \"70f710e7-2590-406b-8520-925636a2c307\" (UID: \"70f710e7-2590-406b-8520-925636a2c307\") "
Sep 14 04:58:03.559096 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.559065    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq" (OuterVolumeSpecName: "kube-api-access-grwgq") pod "70f710e7-2590-406b-8520-925636a2c307" (UID: "70f710e7-2590-406b-8520-925636a2c307"). InnerVolumeSpecName "kube-api-access-grwgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:58:03.658481 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:03.658377    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grwgq\" (UniqueName: \"kubernetes.io/projected/70f710e7-2590-406b-8520-925636a2c307-kube-api-access-grwgq\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:58:04.166757 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.166722    2571 generic.go:356] "Generic (PLEG): container finished" podID="70f710e7-2590-406b-8520-925636a2c307" containerID="2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524" exitCode=0
Sep 14 04:58:04.166757 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.166763    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-544f998b86-4pr4z"
Sep 14 04:58:04.167292 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.166776    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-544f998b86-4pr4z" event={"ID":"70f710e7-2590-406b-8520-925636a2c307","Type":"ContainerDied","Data":"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"}
Sep 14 04:58:04.167292 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.166811    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-544f998b86-4pr4z" event={"ID":"70f710e7-2590-406b-8520-925636a2c307","Type":"ContainerDied","Data":"506897f162da81a634d7e3f453343f718f967013b1d5a277cfb98a183fb4b58d"}
Sep 14 04:58:04.167292 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.166828    2571 scope.go:117] "RemoveContainer" containerID="2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"
Sep 14 04:58:04.176057 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.176039    2571 scope.go:117] "RemoveContainer" containerID="2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"
Sep 14 04:58:04.176363 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:58:04.176345    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524\": container with ID starting with 2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524 not found: ID does not exist" containerID="2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"
Sep 14 04:58:04.176412 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.176373    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524"} err="failed to get container status \"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524\": rpc error: code = NotFound desc = could not find container \"2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524\": container with ID starting with 2ea18fca8e42249195dcfcca3c5abb2a48d3b3be2506d40bf3247b0d2e117524 not found: ID does not exist"
Sep 14 04:58:04.194857 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.194823    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:04.195041 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:04.194889    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-544f998b86-4pr4z"]
Sep 14 04:58:06.193877 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.193843    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70f710e7-2590-406b-8520-925636a2c307" path="/var/lib/kubelet/pods/70f710e7-2590-406b-8520-925636a2c307/volumes"
Sep 14 04:58:06.695089 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.695040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:06.724391 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724360    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:06.724733 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724722    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70f710e7-2590-406b-8520-925636a2c307" containerName="authorino"
Sep 14 04:58:06.724787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724736    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70f710e7-2590-406b-8520-925636a2c307" containerName="authorino"
Sep 14 04:58:06.724787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724747    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f61cae-890f-42cb-8b71-bdaccbc67967" containerName="authorino"
Sep 14 04:58:06.724787 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724752    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f61cae-890f-42cb-8b71-bdaccbc67967" containerName="authorino"
Sep 14 04:58:06.724880 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724830    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70f710e7-2590-406b-8520-925636a2c307" containerName="authorino"
Sep 14 04:58:06.724880 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.724841    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f61cae-890f-42cb-8b71-bdaccbc67967" containerName="authorino"
Sep 14 04:58:06.736023 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.735992    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.738031 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.737915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:06.738474 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.738450    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 04:58:06.738583 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.738450    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 04:58:06.738684 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.738664    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7hwt9\""
Sep 14 04:58:06.739716 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.739689    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 04:58:06.817819 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.817784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:06.886749 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.886716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.886984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.886764    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5v5h\" (UniqueName: \"kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.886984 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.886802    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.893809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.893781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:06.897438 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.897414    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 04:58:06.900918 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.900901    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:06.911552 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.911525    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 04:58:06.987264 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:06.987490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.987490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:06.987490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5v5h\" (UniqueName: \"kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.987490 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987459    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.987705 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.987566    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6f8\" (UniqueName: \"kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:06.990120 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.990093    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.990234 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.990219    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:06.996551 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:06.996514    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5v5h\" (UniqueName: \"kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h\") pod \"maas-controller-d5b57c4ff-r5mlg\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") " pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:07.015495 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.015465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:07.049734 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.049692    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:07.089075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.088625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6f8\" (UniqueName: \"kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.089075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.088690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.089075 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.088744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.091854 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.091827    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.092018 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.091964    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.100023 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.099967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6f8\" (UniqueName: \"kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8\") pod \"maas-controller-6c44665c5c-757ct\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") " pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.212584 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.212547    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:07.325511 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.325484    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:07.328327 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:07.328298    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00444ba1_943a_46a6_a928_33481c7c367c.slice/crio-7ebe152779b8577c926080952318e3982f06028f54a3f1afd8dadff29a038358 WatchSource:0}: Error finding container 7ebe152779b8577c926080952318e3982f06028f54a3f1afd8dadff29a038358: Status 404 returned error can't find the container with id 7ebe152779b8577c926080952318e3982f06028f54a3f1afd8dadff29a038358
Sep 14 04:58:07.491147 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:07.491121    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 04:58:07.493107 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:07.493084    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44552da_2452_4dc5_a431_2ae33baef91d.slice/crio-a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5 WatchSource:0}: Error finding container a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5: Status 404 returned error can't find the container with id a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5
Sep 14 04:58:08.196665 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:08.196627    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-757ct" event={"ID":"f44552da-2452-4dc5-a431-2ae33baef91d","Type":"ContainerStarted","Data":"a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5"}
Sep 14 04:58:08.196841 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:08.196691    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" event={"ID":"00444ba1-943a-46a6-a928-33481c7c367c","Type":"ContainerStarted","Data":"7ebe152779b8577c926080952318e3982f06028f54a3f1afd8dadff29a038358"}
Sep 14 04:58:10.157197 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.157167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:10.206206 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.206166    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-757ct" event={"ID":"f44552da-2452-4dc5-a431-2ae33baef91d","Type":"ContainerStarted","Data":"588764346ff236edd16b4a2f8cb9640082fc6a01a2a43051eb79162a02ad1688"}
Sep 14 04:58:10.206422 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.206332    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:10.207905 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.207875    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" event={"ID":"00444ba1-943a-46a6-a928-33481c7c367c","Type":"ContainerStarted","Data":"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"}
Sep 14 04:58:10.208045 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.207996    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:10.227280 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.227176    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c44665c5c-757ct" podStartSLOduration=1.7402374919999999 podStartE2EDuration="4.227159283s" totalImagesPullingTime="2.486921791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:10.226251582 +0000 UTC m=+684.588204022" watchObservedRunningTime="2026-09-14 04:58:10.227159283 +0000 UTC m=+684.589111725"
Sep 14 04:58:10.232338 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.232312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:10.245812 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:10.245757    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" podStartSLOduration=1.601081907 podStartE2EDuration="4.245737314s" totalImagesPullingTime="2.644655407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:10.24456662 +0000 UTC m=+684.606519064" watchObservedRunningTime="2026-09-14 04:58:10.245737314 +0000 UTC m=+684.607689757"
Sep 14 04:58:21.215418 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.215388    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 04:58:21.215853 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.215456    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:21.287350 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.287308    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:21.287608 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.287523    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" podUID="00444ba1-943a-46a6-a928-33481c7c367c" containerName="manager" containerID="cri-o://752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836" gracePeriod=30
Sep 14 04:58:21.536809 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.536781    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:21.725724 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.725687    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5v5h\" (UniqueName: \"kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h\") pod \"00444ba1-943a-46a6-a928-33481c7c367c\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") "
Sep 14 04:58:21.725724 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.725729    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert\") pod \"00444ba1-943a-46a6-a928-33481c7c367c\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") "
Sep 14 04:58:21.726052 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.725753    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs\") pod \"00444ba1-943a-46a6-a928-33481c7c367c\" (UID: \"00444ba1-943a-46a6-a928-33481c7c367c\") "
Sep 14 04:58:21.728137 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.728102    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "00444ba1-943a-46a6-a928-33481c7c367c" (UID: "00444ba1-943a-46a6-a928-33481c7c367c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:58:21.728137 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.728113    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "00444ba1-943a-46a6-a928-33481c7c367c" (UID: "00444ba1-943a-46a6-a928-33481c7c367c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 04:58:21.728274 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.728195    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h" (OuterVolumeSpecName: "kube-api-access-z5v5h") pod "00444ba1-943a-46a6-a928-33481c7c367c" (UID: "00444ba1-943a-46a6-a928-33481c7c367c"). InnerVolumeSpecName "kube-api-access-z5v5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 04:58:21.827129 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.827025    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5v5h\" (UniqueName: \"kubernetes.io/projected/00444ba1-943a-46a6-a928-33481c7c367c-kube-api-access-z5v5h\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:58:21.827129 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.827065    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-webhook-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:58:21.827129 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:21.827077    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00444ba1-943a-46a6-a928-33481c7c367c-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 04:58:22.279509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.279471    2571 generic.go:356] "Generic (PLEG): container finished" podID="00444ba1-943a-46a6-a928-33481c7c367c" containerID="752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836" exitCode=0
Sep 14 04:58:22.279919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.279528    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg"
Sep 14 04:58:22.279919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.279555    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" event={"ID":"00444ba1-943a-46a6-a928-33481c7c367c","Type":"ContainerDied","Data":"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"}
Sep 14 04:58:22.279919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.279641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d5b57c4ff-r5mlg" event={"ID":"00444ba1-943a-46a6-a928-33481c7c367c","Type":"ContainerDied","Data":"7ebe152779b8577c926080952318e3982f06028f54a3f1afd8dadff29a038358"}
Sep 14 04:58:22.279919 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.279668    2571 scope.go:117] "RemoveContainer" containerID="752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"
Sep 14 04:58:22.288284 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.288268    2571 scope.go:117] "RemoveContainer" containerID="752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"
Sep 14 04:58:22.288570 ip-10-0-137-105 kubenswrapper[2571]: E0914 04:58:22.288548    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836\": container with ID starting with 752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836 not found: ID does not exist" containerID="752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"
Sep 14 04:58:22.288643 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.288578    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836"} err="failed to get container status \"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836\": rpc error: code = NotFound desc = could not find container \"752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836\": container with ID starting with 752e4c86b3cee1ac5300a01c2df3581e6f8395d3de12263b81db5f31e737e836 not found: ID does not exist"
Sep 14 04:58:22.297339 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.297314    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:22.301446 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:22.301421    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d5b57c4ff-r5mlg"]
Sep 14 04:58:24.194566 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:24.194527    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00444ba1-943a-46a6-a928-33481c7c367c" path="/var/lib/kubelet/pods/00444ba1-943a-46a6-a928-33481c7c367c/volumes"
Sep 14 04:58:38.786631 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:38.786556    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:41.764505 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:41.764460    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:41.871444 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:41.871411    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.276103 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.276070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.375396 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.375367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.419142 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.419109    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 04:58:43.419513 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.419501    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00444ba1-943a-46a6-a928-33481c7c367c" containerName="manager"
Sep 14 04:58:43.419558 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.419514    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="00444ba1-943a-46a6-a928-33481c7c367c" containerName="manager"
Sep 14 04:58:43.419594 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.419580    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="00444ba1-943a-46a6-a928-33481c7c367c" containerName="manager"
Sep 14 04:58:43.427089 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.427056    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.429914 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.429878    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 04:58:43.430109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.429922    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m9nv8\""
Sep 14 04:58:43.430109 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.429979    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 04:58:43.434293 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.434267    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 04:58:43.503772 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.503729    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.526044 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.526011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.526203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.526071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.526203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.526110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.526203 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.526133    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghn8\" (UniqueName: \"kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.604081 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.604036    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.627281 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.627247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.627476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.627305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.627476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.627344    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghn8\" (UniqueName: \"kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.627476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.627422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.628152 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.628072    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.630089 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.630067    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.630212 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.630191    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.636550 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.636515    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghn8\" (UniqueName: \"kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8\") pod \"maas-api-5f795d4696-826mf\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") " pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.739864 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.739761    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:43.788856 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.788747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:43.903405 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:43.903366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:44.009777 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.009744    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"]
Sep 14 04:58:44.014036 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.014016    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.016420 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.016396    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-99mff\""
Sep 14 04:58:44.016542 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.016398    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 04:58:44.023341 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.023172    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"]
Sep 14 04:58:44.026177 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.026153    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 04:58:44.029117 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:44.029079    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454f9f2a_f35c_4658_bd10_5487c8751089.slice/crio-6b9cd12df8e98c10d883edfaaa359686f354ac873273281eb338533b3475b320 WatchSource:0}: Error finding container 6b9cd12df8e98c10d883edfaaa359686f354ac873273281eb338533b3475b320: Status 404 returned error can't find the container with id 6b9cd12df8e98c10d883edfaaa359686f354ac873273281eb338533b3475b320
Sep 14 04:58:44.131251 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.131127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxw6q\" (UniqueName: \"kubernetes.io/projected/cda02c30-fd51-4775-9881-11d9b3493245-kube-api-access-rxw6q\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.131251 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.131216    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cda02c30-fd51-4775-9881-11d9b3493245-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.231831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.231786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxw6q\" (UniqueName: \"kubernetes.io/projected/cda02c30-fd51-4775-9881-11d9b3493245-kube-api-access-rxw6q\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.231831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.231833    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cda02c30-fd51-4775-9881-11d9b3493245-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.232546 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.232522    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cda02c30-fd51-4775-9881-11d9b3493245-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.240253 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.240216    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxw6q\" (UniqueName: \"kubernetes.io/projected/cda02c30-fd51-4775-9881-11d9b3493245-kube-api-access-rxw6q\") pod \"payload-pre-processing-59fbc47bf8-gtt2g\" (UID: \"cda02c30-fd51-4775-9881-11d9b3493245\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.327903 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.327867    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:44.394102 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.394030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" event={"ID":"454f9f2a-f35c-4658-bd10-5487c8751089","Type":"ContainerStarted","Data":"6b9cd12df8e98c10d883edfaaa359686f354ac873273281eb338533b3475b320"}
Sep 14 04:58:44.603630 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:44.603602    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"]
Sep 14 04:58:44.605903 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:58:44.605873    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda02c30_fd51_4775_9881_11d9b3493245.slice/crio-0058f5dd8293e850e198ad783d469e67e4d610e409b9323bedccf6a02fe62e7f WatchSource:0}: Error finding container 0058f5dd8293e850e198ad783d469e67e4d610e409b9323bedccf6a02fe62e7f: Status 404 returned error can't find the container with id 0058f5dd8293e850e198ad783d469e67e4d610e409b9323bedccf6a02fe62e7f
Sep 14 04:58:45.402831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:45.402789    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g" event={"ID":"cda02c30-fd51-4775-9881-11d9b3493245","Type":"ContainerStarted","Data":"0058f5dd8293e850e198ad783d469e67e4d610e409b9323bedccf6a02fe62e7f"}
Sep 14 04:58:46.410299 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:46.410260    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" event={"ID":"454f9f2a-f35c-4658-bd10-5487c8751089","Type":"ContainerStarted","Data":"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"}
Sep 14 04:58:46.410727 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:46.410376    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:48.422049 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:48.422010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g" event={"ID":"cda02c30-fd51-4775-9881-11d9b3493245","Type":"ContainerStarted","Data":"425b6cc5089c2e66ff2f512d55e0070d7d8259bbc3276525111accbe0e89739d"}
Sep 14 04:58:48.422460 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:48.422156    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:48.443509 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:48.441301    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" podStartSLOduration=3.884325735 podStartE2EDuration="5.441277858s" totalImagesPullingTime="1.556952123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:46.434877529 +0000 UTC m=+720.796830159" watchObservedRunningTime="2026-09-14 04:58:48.441277858 +0000 UTC m=+722.803230302"
Sep 14 04:58:48.444506 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:48.443907    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g" podStartSLOduration=2.6758896549999998 podStartE2EDuration="5.443891074s" totalImagesPullingTime="2.768001419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:58:48.438441994 +0000 UTC m=+722.800394436" watchObservedRunningTime="2026-09-14 04:58:48.443891074 +0000 UTC m=+722.805843517"
Sep 14 04:58:52.421433 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:52.421403    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 04:58:53.444831 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:53.444791    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:53.542651 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:53.542613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:59.428552 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.428517    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gtt2g"
Sep 14 04:58:59.841343 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.841311    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"]
Sep 14 04:58:59.845324 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.845301    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.847807 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.847781    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 04:58:59.847979 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.847786    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c7z9x\""
Sep 14 04:58:59.847979 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.847791    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 04:58:59.848811 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.848791    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 14 04:58:59.855532 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.855504    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"]
Sep 14 04:58:59.862602 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.862577    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.982999    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.983060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.983121    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.983226    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.983278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:58:59.983465 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:58:59.983302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wchp\" (UniqueName: \"kubernetes.io/projected/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-kube-api-access-2wchp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084226 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084226 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wchp\" (UniqueName: \"kubernetes.io/projected/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-kube-api-access-2wchp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084267    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084297    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084476 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084688 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084670    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.084743 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.084717    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.085066 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.085042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.086703 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.086682    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.087072 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.087052    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.094014 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.093913    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wchp\" (UniqueName: \"kubernetes.io/projected/c8a4f8ab-394b-4294-b0c6-b9b8b864830a-kube-api-access-2wchp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh\" (UID: \"c8a4f8ab-394b-4294-b0c6-b9b8b864830a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.156811 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.156769    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:00.432401 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.432314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"]
Sep 14 04:59:00.436147 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:59:00.436111    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a4f8ab_394b_4294_b0c6_b9b8b864830a.slice/crio-ce3e1638b8d5435f0b15375baa67b90bf2e79363d2298da25375dec7a9feca56 WatchSource:0}: Error finding container ce3e1638b8d5435f0b15375baa67b90bf2e79363d2298da25375dec7a9feca56: Status 404 returned error can't find the container with id ce3e1638b8d5435f0b15375baa67b90bf2e79363d2298da25375dec7a9feca56
Sep 14 04:59:00.491130 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:00.491088    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh" event={"ID":"c8a4f8ab-394b-4294-b0c6-b9b8b864830a","Type":"ContainerStarted","Data":"ce3e1638b8d5435f0b15375baa67b90bf2e79363d2298da25375dec7a9feca56"}
Sep 14 04:59:02.504675 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:02.504637    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh" event={"ID":"c8a4f8ab-394b-4294-b0c6-b9b8b864830a","Type":"ContainerStarted","Data":"63bdecfe8d9644d6d7038d22a3a30f00f038c69c22ed190cf572779558d9664f"}
Sep 14 04:59:02.524639 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:02.524588    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh" podStartSLOduration=2.344704338 podStartE2EDuration="3.524572866s" totalImagesPullingTime="1.179868528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:59:02.523524768 +0000 UTC m=+736.885477223" watchObservedRunningTime="2026-09-14 04:59:02.524572866 +0000 UTC m=+736.886525308"
Sep 14 04:59:04.110701 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:04.110661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:09.449690 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:09.449643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:10.157866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:10.157817    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:10.158157 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:10.157889    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:10.171486 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:10.171441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:10.557742 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:10.557708    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh"
Sep 14 04:59:12.516887 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:12.516846    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:18.723605 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:18.723563    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:24.705250 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:24.705211    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:40.237494 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.237450    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"]
Sep 14 04:59:40.241201 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.241180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.243990 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.243964    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 04:59:40.253971 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.253924    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"]
Sep 14 04:59:40.262317 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.262282    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:40.351866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.351828    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.351866 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.351865    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t72j\" (UniqueName: \"kubernetes.io/projected/a85ecbf3-8081-4a0f-a87f-86e673ead417-kube-api-access-7t72j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.352166 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.351887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.352166 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.351911    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a85ecbf3-8081-4a0f-a87f-86e673ead417-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.352166 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.351988    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.352166 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.352062    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.452578 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452533    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.452578 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t72j\" (UniqueName: \"kubernetes.io/projected/a85ecbf3-8081-4a0f-a87f-86e673ead417-kube-api-access-7t72j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.452846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.452846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452614    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a85ecbf3-8081-4a0f-a87f-86e673ead417-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.452846 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452817    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.453118 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.452866    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.453118 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.453045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.453118 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.453105    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.453260 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.453239    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.455162 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.455132    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a85ecbf3-8081-4a0f-a87f-86e673ead417-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.455371 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.455350    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a85ecbf3-8081-4a0f-a87f-86e673ead417-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.462576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.462548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t72j\" (UniqueName: \"kubernetes.io/projected/a85ecbf3-8081-4a0f-a87f-86e673ead417-kube-api-access-7t72j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vqhh6\" (UID: \"a85ecbf3-8081-4a0f-a87f-86e673ead417\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.553576 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.553470    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:40.831803 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:40.831774    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"]
Sep 14 04:59:40.834018 ip-10-0-137-105 kubenswrapper[2571]: W0914 04:59:40.833990    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85ecbf3_8081_4a0f_a87f_86e673ead417.slice/crio-0b1ed8085627e2c08abd9ca5a7746b01b3415feb596f9de6681c49ec0b9f8e8d WatchSource:0}: Error finding container 0b1ed8085627e2c08abd9ca5a7746b01b3415feb596f9de6681c49ec0b9f8e8d: Status 404 returned error can't find the container with id 0b1ed8085627e2c08abd9ca5a7746b01b3415feb596f9de6681c49ec0b9f8e8d
Sep 14 04:59:41.708315 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:41.708271    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6" event={"ID":"a85ecbf3-8081-4a0f-a87f-86e673ead417","Type":"ContainerStarted","Data":"5c9339e8912956ec4d736ccadca8ed46abdcfa07eeaa7aeb2c9a0e1a78c97d89"}
Sep 14 04:59:41.708731 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:41.708332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6" event={"ID":"a85ecbf3-8081-4a0f-a87f-86e673ead417","Type":"ContainerStarted","Data":"0b1ed8085627e2c08abd9ca5a7746b01b3415feb596f9de6681c49ec0b9f8e8d"}
Sep 14 04:59:41.729064 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:41.729007    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6" podStartSLOduration=1.534303448 podStartE2EDuration="1.728987779s" totalImagesPullingTime="194.684331ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 04:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 04:59:41.727098173 +0000 UTC m=+776.089050616" watchObservedRunningTime="2026-09-14 04:59:41.728987779 +0000 UTC m=+776.090940222"
Sep 14 04:59:47.170163 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:47.170117    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 04:59:50.554377 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:50.554338    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:50.554779 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:50.554405    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:50.567525 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:50.567497    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:50.769350 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:50.769319    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vqhh6"
Sep 14 04:59:54.955486 ip-10-0-137-105 kubenswrapper[2571]: I0914 04:59:54.955447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:00:03.776266 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:00:03.776229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:04.666460 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:04.666426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:04.790710 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:04.790670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:05.885030 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:05.884989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:05.985166 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:05.985129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.164473 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.164437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.266282 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.266244    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.448876 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.448785    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.486607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.486573    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:01:28.490520 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.490494    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.493222 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.493199    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 05:01:28.493375 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.493201    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-c366d3-openshift-default-dockercfg-b26hf\""
Sep 14 05:01:28.502968 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.502922    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:01:28.526886 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.526842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.596530 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.596498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.643121 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643082    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643121 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.643466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.643410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46h89\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.657046 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.657011    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.744788 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744746    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.744788 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744788    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744810    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744867    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.744968    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745011    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46h89\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745464 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745253    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745464 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745328    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745569 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745478    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745635 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.745869 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.745845    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.747477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.747454    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.747891 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.747867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.753207 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.753179    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.753471 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.753454    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46h89\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89\") pod \"e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") " pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.798942 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.798902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:28.804882 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.804865    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:28.994607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:28.994570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:29.081216 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:29.081184    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:01:29.083130 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:29.083097    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece0edeb_0aeb_4a1b_a3dc_0745008115b2.slice/crio-fb9f16647bf9b91126b34d1fe7ea5e2f78bdd87a862eb4b06296262ffb01f211 WatchSource:0}: Error finding container fb9f16647bf9b91126b34d1fe7ea5e2f78bdd87a862eb4b06296262ffb01f211: Status 404 returned error can't find the container with id fb9f16647bf9b91126b34d1fe7ea5e2f78bdd87a862eb4b06296262ffb01f211
Sep 14 05:01:29.089541 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:29.089511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:29.275304 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:29.275268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerStarted","Data":"fb9f16647bf9b91126b34d1fe7ea5e2f78bdd87a862eb4b06296262ffb01f211"}
Sep 14 05:01:29.646637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:29.646584    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:29.749681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:29.749637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:30.157361 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:30.157271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:31.620844 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:31.620802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:31.717704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:31.717665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:31.820416 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:31.820369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:32.002758 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.002718    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:32.002827 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.002799    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:32.002873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.002828    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:32.295055 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.294960    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerStarted","Data":"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"}
Sep 14 05:01:32.316088 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.316018    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podStartSLOduration=1.398808619 podStartE2EDuration="4.315995706s" totalImagesPullingTime="2.917187087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:32.315308217 +0000 UTC m=+886.677260660" watchObservedRunningTime="2026-09-14 05:01:32.315995706 +0000 UTC m=+886.677948150"
Sep 14 05:01:32.805450 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.805410    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:32.829462 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.829427    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:01:32.833883 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.833862    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:32.836378 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.836355    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-667664-serving-cert\""
Sep 14 05:01:32.845031 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.844778    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:01:32.862720 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.862688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:32.962240 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.962210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:32.990709 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.990669    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:32.990709 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.990712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:32.990991 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.990742    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwt8\" (UniqueName: \"kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:32.990991 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:32.990882    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.062595 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.062505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:33.091805 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.091763    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.091805 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.091806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.092120 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.091836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwt8\" (UniqueName: \"kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.092120 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.091891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.092652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.092629    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.094567 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.094546    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.094649 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.094603    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.100122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.100099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwt8\" (UniqueName: \"kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8\") pod \"maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.147811 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.147774    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:33.438800 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.438773    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:01:33.441021 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:33.440992    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4a2b79_fa56_479c_a99b_850045848771.slice/crio-0d5f269eaa153a3e961ecbfa4d508d5737175b8257c5df8000ee61f176e767a7 WatchSource:0}: Error finding container 0d5f269eaa153a3e961ecbfa4d508d5737175b8257c5df8000ee61f176e767a7: Status 404 returned error can't find the container with id 0d5f269eaa153a3e961ecbfa4d508d5737175b8257c5df8000ee61f176e767a7
Sep 14 05:01:33.573221 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.573175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:33.675882 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.675823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:33.815835 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:33.815807    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:34.027709 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.027675    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:34.236428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.236395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:34.307830 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.307790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:34.309263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.309232    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" event={"ID":"fd4a2b79-fa56-479c-a99b-850045848771","Type":"ContainerStarted","Data":"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"}
Sep 14 05:01:34.309416 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.309294    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" event={"ID":"fd4a2b79-fa56-479c-a99b-850045848771","Type":"ContainerStarted","Data":"0d5f269eaa153a3e961ecbfa4d508d5737175b8257c5df8000ee61f176e767a7"}
Sep 14 05:01:34.309586 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.309562    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:34.309675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.309638    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:34.310405 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.310390    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:34.327821 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.327765    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" podStartSLOduration=1.894348618 podStartE2EDuration="2.327745053s" totalImagesPullingTime="433.396435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:34.32619067 +0000 UTC m=+888.688143123" watchObservedRunningTime="2026-09-14 05:01:34.327745053 +0000 UTC m=+888.689697496"
Sep 14 05:01:34.997895 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:34.997857    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:01:35.002637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.002611    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.005571 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.005549    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-e08402-serving-cert\""
Sep 14 05:01:35.013059 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.013029    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2pbf\" (UniqueName: \"kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.013287 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.013260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.013404 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.013325    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.013404 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.013360    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.016420 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.016394    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:01:35.024717 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.024679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:35.111321 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.111283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:35.114487 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.114461    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2pbf\" (UniqueName: \"kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.114617 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.114527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.114617 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.114578    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.114617 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.114614    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.114768 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:35.114731    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-serving-cert: secret "maas-api-e2e-worker-w4-e08402-serving-cert" not found
Sep 14 05:01:35.114822 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:35.114800    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls podName:5083fffa-b380-4ddb-a3b2-e6e46e51d965 nodeName:}" failed. No retries permitted until 2026-09-14 05:01:35.614783538 +0000 UTC m=+889.976735958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls") pod "maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" (UID: "5083fffa-b380-4ddb-a3b2-e6e46e51d965") : secret "maas-api-e2e-worker-w4-e08402-serving-cert" not found
Sep 14 05:01:35.115305 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.115282    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.117146 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.117128    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.122672 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.122646    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2pbf\" (UniqueName: \"kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.311735 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.311642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:35.610156 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.610070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:35.619318 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.619259    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.622222 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.622194    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:35.702572 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.702533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:35.915088 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:35.914997    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:36.192960 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:36.192914    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5083fffa_b380_4ddb_a3b2_e6e46e51d965.slice/crio-6382a45cad0117f445e92f6d2c83a2ae7d12f358024d63d2f8e1339916074953 WatchSource:0}: Error finding container 6382a45cad0117f445e92f6d2c83a2ae7d12f358024d63d2f8e1339916074953: Status 404 returned error can't find the container with id 6382a45cad0117f445e92f6d2c83a2ae7d12f358024d63d2f8e1339916074953
Sep 14 05:01:36.197217 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.197195    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:01:36.223565 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.223418    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:01:36.228785 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.228758    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.231102 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.231079    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-c366d3-serving-cert\""
Sep 14 05:01:36.237085 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.236490    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:01:36.275999 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.275957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:36.320997 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.320952    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" event={"ID":"5083fffa-b380-4ddb-a3b2-e6e46e51d965","Type":"ContainerStarted","Data":"6382a45cad0117f445e92f6d2c83a2ae7d12f358024d63d2f8e1339916074953"}
Sep 14 05:01:36.325100 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.325074    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.325244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.325126    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.325244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.325156    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87gtw\" (UniqueName: \"kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.325244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.325222    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.388491 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.388396    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:36.426307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.426274    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.426499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.426364    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.426499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.426393    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87gtw\" (UniqueName: \"kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.426499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.426442    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.426644 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:36.426611    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-serving-cert: secret "maas-api-e2e-worker-w0-c366d3-serving-cert" not found
Sep 14 05:01:36.426719 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:36.426697    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls podName:db618531-409a-4734-976b-a2576462af31 nodeName:}" failed. No retries permitted until 2026-09-14 05:01:36.926674414 +0000 UTC m=+891.288626837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls") pod "maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" (UID: "db618531-409a-4734-976b-a2576462af31") : secret "maas-api-e2e-worker-w0-c366d3-serving-cert" not found
Sep 14 05:01:36.427198 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.427178    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.429147 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.429124    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.437757 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.437730    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87gtw\" (UniqueName: \"kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.561681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.561650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:36.834386 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.834262    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:36.932149 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.932112    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:36.932968 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.932917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:36.934707 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:36.934685    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") pod \"maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj\" (UID: \"db618531-409a-4734-976b-a2576462af31\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:37.036005 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.035962    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:01:37.039574 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.039557    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.042170 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.042142    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-c366d3-dockercfg-vv4f4\""
Sep 14 05:01:37.042317 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.042287    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-c366d3\""
Sep 14 05:01:37.048988 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.048963    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:01:37.133965 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.133794    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwblk\" (UniqueName: \"kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.133965 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.133857    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.146675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.146642    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:37.234684 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.234641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwblk\" (UniqueName: \"kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.235188 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.234722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.235700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.235676    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.246732 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.246699    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwblk\" (UniqueName: \"kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk\") pod \"payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.267586 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.267547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:37.327763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.327727    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" event={"ID":"5083fffa-b380-4ddb-a3b2-e6e46e51d965","Type":"ContainerStarted","Data":"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"}
Sep 14 05:01:37.327991 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.327860    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:37.348108 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.348048    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" podStartSLOduration=2.999068456 podStartE2EDuration="3.348030486s" totalImagesPullingTime="348.96203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:37.346471807 +0000 UTC m=+891.708424254" watchObservedRunningTime="2026-09-14 05:01:37.348030486 +0000 UTC m=+891.709982937"
Sep 14 05:01:37.352828 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.352803    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:37.430424 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.430399    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:01:37.432481 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:37.432450    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb618531_409a_4734_976b_a2576462af31.slice/crio-fe479baea4d0afcb2060dd213a10f154d799ef4ee0466b9a3b77210bbd349fb2 WatchSource:0}: Error finding container fe479baea4d0afcb2060dd213a10f154d799ef4ee0466b9a3b77210bbd349fb2: Status 404 returned error can't find the container with id fe479baea4d0afcb2060dd213a10f154d799ef4ee0466b9a3b77210bbd349fb2
Sep 14 05:01:37.632016 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.631979    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:01:37.633768 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:37.633735    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda6b144e_a407_430a_9d77_9632e193c663.slice/crio-a52d0500d4f444569e84dfeb48ba7f094a530d1df2462c2d8b1e45fb9ece2363 WatchSource:0}: Error finding container a52d0500d4f444569e84dfeb48ba7f094a530d1df2462c2d8b1e45fb9ece2363: Status 404 returned error can't find the container with id a52d0500d4f444569e84dfeb48ba7f094a530d1df2462c2d8b1e45fb9ece2363
Sep 14 05:01:37.652525 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.652495    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:37.751253 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.751217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:37.866317 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:37.866283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:38.149488 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.149447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:38.308408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.308374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:38.335944 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.335904    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" event={"ID":"da6b144e-a407-430a-9d77-9632e193c663","Type":"ContainerStarted","Data":"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"}
Sep 14 05:01:38.336122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.335973    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" event={"ID":"da6b144e-a407-430a-9d77-9632e193c663","Type":"ContainerStarted","Data":"a52d0500d4f444569e84dfeb48ba7f094a530d1df2462c2d8b1e45fb9ece2363"}
Sep 14 05:01:38.336122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.336064    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:38.337387 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.337353    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" event={"ID":"db618531-409a-4734-976b-a2576462af31","Type":"ContainerStarted","Data":"fe479baea4d0afcb2060dd213a10f154d799ef4ee0466b9a3b77210bbd349fb2"}
Sep 14 05:01:38.354397 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.354336    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" podStartSLOduration=1.3543186600000001 podStartE2EDuration="1.35431866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:38.353298455 +0000 UTC m=+892.715250903" watchObservedRunningTime="2026-09-14 05:01:38.35431866 +0000 UTC m=+892.716271100"
Sep 14 05:01:38.639437 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:38.639404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:39.028758 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:39.028728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:39.344734 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:39.344631    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" event={"ID":"db618531-409a-4734-976b-a2576462af31","Type":"ContainerStarted","Data":"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"}
Sep 14 05:01:39.345204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:39.344800    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:39.365627 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:39.365577    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" podStartSLOduration=2.316216508 podStartE2EDuration="3.365559104s" totalImagesPullingTime="1.049342596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:39.364274439 +0000 UTC m=+893.726226893" watchObservedRunningTime="2026-09-14 05:01:39.365559104 +0000 UTC m=+893.727511547"
Sep 14 05:01:39.833865 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:39.833831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:40.320380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:40.320352    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:01:40.561574 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:40.561544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:40.943248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:40.943207    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:41.043317 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:41.043277    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:41.509327 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:41.509297    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:42.562556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:42.562521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:43.343980 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:43.343949    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:01:43.383246 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:43.383180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:44.329083 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.329049    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:01:44.333503 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.333472    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.336159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.336130    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-654e8bde-serving-cert\""
Sep 14 05:01:44.344346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.343621    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:01:44.378885 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.378735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:44.413425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.413390    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.413624 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.413435    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfvq2\" (UniqueName: \"kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.413624 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.413472    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.413624 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.413537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.434140 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.434102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:44.514744 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.514703    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.514744 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.514750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfvq2\" (UniqueName: \"kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.515003 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.514792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.515003 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.514815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.515003 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:44.514982    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-serving-cert: secret "maas-api-e2e-ait-654e8bde-serving-cert" not found
Sep 14 05:01:44.515118 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:44.515055    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls podName:c114b06b-955d-4a49-bbd3-5b559d11ca9b nodeName:}" failed. No retries permitted until 2026-09-14 05:01:45.015036408 +0000 UTC m=+899.376988829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls") pod "maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" (UID: "c114b06b-955d-4a49-bbd3-5b559d11ca9b") : secret "maas-api-e2e-ait-654e8bde-serving-cert" not found
Sep 14 05:01:44.515576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.515544    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.517603 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.517580    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.523377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.523345    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfvq2\" (UniqueName: \"kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:44.597408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.597313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:44.899609 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.899458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:44.999348 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:44.999314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:45.021095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.021061    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:45.024589 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.024550    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") pod \"maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:45.248777 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.248727    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:45.356421 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.356393    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:01:45.547322 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.547227    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:45.555729 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:45.555704    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:01:45.558057 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:45.557960    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc114b06b_955d_4a49_bbd3_5b559d11ca9b.slice/crio-f5914b628b8d3edd745278283bef401fb235135a9bf87958ff6faf2c7d42574d WatchSource:0}: Error finding container f5914b628b8d3edd745278283bef401fb235135a9bf87958ff6faf2c7d42574d: Status 404 returned error can't find the container with id f5914b628b8d3edd745278283bef401fb235135a9bf87958ff6faf2c7d42574d
Sep 14 05:01:46.151463 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.151438    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:01:46.153074 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.153051    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:01:46.158061 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.158038    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:01:46.159371 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.159348    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:01:46.378601 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.378505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:46.419249 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.419213    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" event={"ID":"c114b06b-955d-4a49-bbd3-5b559d11ca9b","Type":"ContainerStarted","Data":"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"}
Sep 14 05:01:46.419439 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.419274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" event={"ID":"c114b06b-955d-4a49-bbd3-5b559d11ca9b","Type":"ContainerStarted","Data":"f5914b628b8d3edd745278283bef401fb235135a9bf87958ff6faf2c7d42574d"}
Sep 14 05:01:46.419439 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.419305    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:46.438561 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.438512    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" podStartSLOduration=1.972359407 podStartE2EDuration="2.438494134s" totalImagesPullingTime="466.134727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:46.436751738 +0000 UTC m=+900.798704228" watchObservedRunningTime="2026-09-14 05:01:46.438494134 +0000 UTC m=+900.800446577"
Sep 14 05:01:46.441391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.441355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:46.715760 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:46.715720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:47.025394 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:47.025300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:47.249779 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:47.249742    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:47.349920 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:47.349817    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:48.160468 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:48.160412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:48.783712 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:48.783672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:48.888371 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:48.888325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:48.984206 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:48.984168    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:49.084809 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:49.084715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:49.249774 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:49.249744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:49.347640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:49.347548    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:01:49.836876 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:49.836839    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:50.351104 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:50.351068    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 05:01:50.378488 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:50.351128    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 05:01:50.695713 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:50.695614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.138561 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.138522    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.236921 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.236890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.451421 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.451335    2571 generic.go:356] "Generic (PLEG): container finished" podID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerID="370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0" exitCode=0
Sep 14 05:01:51.451810 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.451415    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerDied","Data":"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"}
Sep 14 05:01:51.451880 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.451823    2571 scope.go:117] "RemoveContainer" containerID="370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"
Sep 14 05:01:51.453161 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.453131    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:51.453264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.453207    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:51.453264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.453253    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 14 05:01:51.508115 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.508085    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.673687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.673648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.853160 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.853126    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:51.961607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:51.961544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:52.060581 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.060550    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:52.160895 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.160808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:52.431455 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.431373    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:01:52.460311 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.460272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerStarted","Data":"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"}
Sep 14 05:01:52.772402 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.772355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:52.805447 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.805406    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:52.805621 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.805512    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 05:01:52.805621 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:52.805560    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 05:01:53.546379 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:53.546342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:53.650672 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:53.650639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:53.805406 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:53.805314    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 05:01:53.805406 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:53.805368    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 05:01:54.058506 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.058399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.182065 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.182032    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.535547 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.535511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.634157 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.634111    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.734518 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.734485    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.765253 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.765216    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:01:54.783276 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.783243    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.793982 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.793877    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e7fdd6-serving-cert\""
Sep 14 05:01:54.797002 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.795651    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:01:54.810049 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.809899    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 05:01:54.810049 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.809990    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 05:01:54.813901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.813833    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.824164 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.824134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.824322 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.824275    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.824366 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.824347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgf5s\" (UniqueName: \"kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.824432 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.824374    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.866861 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.866822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:54.925753 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.925716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.925980 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.925823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.925980 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.925892    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgf5s\" (UniqueName: \"kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.926101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.925979    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.926101 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:54.925995    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-serving-cert: secret "maas-api-e2e-inf-b-e7fdd6-serving-cert" not found
Sep 14 05:01:54.926101 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:01:54.926069    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls podName:e550c7d0-4ebd-45a9-a42b-5c363afdc266 nodeName:}" failed. No retries permitted until 2026-09-14 05:01:55.426052771 +0000 UTC m=+909.788005191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls") pod "maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" (UID: "e550c7d0-4ebd-45a9-a42b-5c363afdc266") : secret "maas-api-e2e-inf-b-e7fdd6-serving-cert" not found
Sep 14 05:01:54.926757 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.926726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.928514 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.928492    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:54.935536 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:54.935503    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgf5s\" (UniqueName: \"kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:55.070267 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.070151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:55.171495 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.171459    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:55.432148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.432051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:55.435489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.435457    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") pod \"maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:55.478380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.478346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:55.583155 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.583114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:55.715330 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.713526    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:55.810955 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.810560    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 05:01:55.810955 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:55.810707    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 05:01:56.019663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:56.019613    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:01:56.022948 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:01:56.022823    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode550c7d0_4ebd_45a9_a42b_5c363afdc266.slice/crio-b1b52a888e187809155863751269778f4efb43144909749cf160eaa0659506c0 WatchSource:0}: Error finding container b1b52a888e187809155863751269778f4efb43144909749cf160eaa0659506c0: Status 404 returned error can't find the container with id b1b52a888e187809155863751269778f4efb43144909749cf160eaa0659506c0
Sep 14 05:01:56.060792 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:56.060746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:56.514348 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:56.514313    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" event={"ID":"e550c7d0-4ebd-45a9-a42b-5c363afdc266","Type":"ContainerStarted","Data":"b1b52a888e187809155863751269778f4efb43144909749cf160eaa0659506c0"}
Sep 14 05:01:56.805979 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:56.805921    2571 patch_prober.go:28] interesting pod/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 05:01:56.806402 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:56.805998    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 05:01:57.158887 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.158809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:57.521735 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.521691    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" event={"ID":"e550c7d0-4ebd-45a9-a42b-5c363afdc266","Type":"ContainerStarted","Data":"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"}
Sep 14 05:01:57.522451 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.522426    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:01:57.540071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.540018    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" podStartSLOduration=2.765778445 podStartE2EDuration="3.540001814s" totalImagesPullingTime="774.223369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:01:57.539221442 +0000 UTC m=+911.901173886" watchObservedRunningTime="2026-09-14 05:01:57.540001814 +0000 UTC m=+911.901954256"
Sep 14 05:01:57.823072 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.822976    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:57.823641 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.823322    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:57.825567 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.825491    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:01:57.981377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:57.981322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:58.081761 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:58.081677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:58.185033 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:58.184993    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:58.282280 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:58.282245    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:58.878156 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:58.878117    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:01:59.865538 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:01:59.865505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:00.223204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:00.223167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:00.325752 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:00.325716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:00.854157 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:00.854117    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:01.561136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:01.561096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:02.273005 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.272973    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:02:02.281629 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.281588    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.284380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.284350    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-f14475-serving-cert\""
Sep 14 05:02:02.286741 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.286715    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:02:02.332747 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.332716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:02.403312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.403279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.403312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.403313    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8679l\" (UniqueName: \"kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.403558 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.403338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.403558 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.403499    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.430509 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.430472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:02.505034 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.504999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.505255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.505106    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.505255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.505131    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8679l\" (UniqueName: \"kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.505255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.505154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.505431 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:02.505335    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-f14475-serving-cert: secret "maas-api-e2e-autores-f14475-serving-cert" not found
Sep 14 05:02:02.505431 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:02.505406    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls podName:e9bf71b6-5da7-4b39-9c3f-2cad5f01d614 nodeName:}" failed. No retries permitted until 2026-09-14 05:02:03.005384712 +0000 UTC m=+917.367337135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls") pod "maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" (UID: "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614") : secret "maas-api-e2e-autores-f14475-serving-cert" not found
Sep 14 05:02:02.505845 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.505826    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.508171 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.508138    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.516121 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.516097    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8679l\" (UniqueName: \"kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:02.847133 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.847096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:02.943095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:02.943058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:03.008872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.008831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:03.011641 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.011610    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") pod \"maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:03.014724 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.014695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:03.197060 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.196970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:03.346706 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.346666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:03.479585 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.479551    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:02:03.481893 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:02:03.481857    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9bf71b6_5da7_4b39_9c3f_2cad5f01d614.slice/crio-d6adbb420202676f812cc6f363ed35520aef8b360108d4d62bcd019faf2ea8c2 WatchSource:0}: Error finding container d6adbb420202676f812cc6f363ed35520aef8b360108d4d62bcd019faf2ea8c2: Status 404 returned error can't find the container with id d6adbb420202676f812cc6f363ed35520aef8b360108d4d62bcd019faf2ea8c2
Sep 14 05:02:03.555913 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:03.555876    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" event={"ID":"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614","Type":"ContainerStarted","Data":"d6adbb420202676f812cc6f363ed35520aef8b360108d4d62bcd019faf2ea8c2"}
Sep 14 05:02:04.170031 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.169965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:04.538276 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.538247    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:02:04.563866 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.563828    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" event={"ID":"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614","Type":"ContainerStarted","Data":"652d5c8a9326d0fe4ad45d95f8b2bf442a5c71fcad7c73213b73eeaa18f9ccd9"}
Sep 14 05:02:04.564080 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.564021    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:04.585286 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.585218    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" podStartSLOduration=2.031867635 podStartE2EDuration="2.585190797s" totalImagesPullingTime="553.323162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:02:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:02:04.582157226 +0000 UTC m=+918.944109682" watchObservedRunningTime="2026-09-14 05:02:04.585190797 +0000 UTC m=+918.947143241"
Sep 14 05:02:04.612478 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.612442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:04.736068 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.736039    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:04.839144 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:04.839054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:05.007893 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:05.007854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:05.129436 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:05.129345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:05.954814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:05.954781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:06.659768 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:06.659732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.307947 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.307889    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:02:07.308409 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.308150    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerName="maas-api" containerID="cri-o://d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf" gracePeriod=30
Sep 14 05:02:07.339513 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.339482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.426159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.426124    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.57:8443/health\": dial tcp 10.134.0.57:8443: connect: connection refused"
Sep 14 05:02:07.440340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.440301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.531409 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.531371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.582165 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.582129    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:02:07.584285 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.584255    2571 generic.go:356] "Generic (PLEG): container finished" podID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerID="d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf" exitCode=0
Sep 14 05:02:07.584467 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.584296    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"
Sep 14 05:02:07.584467 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.584335    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" event={"ID":"c114b06b-955d-4a49-bbd3-5b559d11ca9b","Type":"ContainerDied","Data":"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"}
Sep 14 05:02:07.584467 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.584400    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65" event={"ID":"c114b06b-955d-4a49-bbd3-5b559d11ca9b","Type":"ContainerDied","Data":"f5914b628b8d3edd745278283bef401fb235135a9bf87958ff6faf2c7d42574d"}
Sep 14 05:02:07.584467 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.584431    2571 scope.go:117] "RemoveContainer" containerID="d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"
Sep 14 05:02:07.586227 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.586148    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.594714 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.594687    2571 scope.go:117] "RemoveContainer" containerID="d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"
Sep 14 05:02:07.595130 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:07.595104    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf\": container with ID starting with d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf not found: ID does not exist" containerID="d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"
Sep 14 05:02:07.595231 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.595144    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf"} err="failed to get container status \"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf\": rpc error: code = NotFound desc = could not find container \"d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf\": container with ID starting with d46f7122d4bea40756fcbf47fe09576104e21d3a6fd1fc1d10f0c38b5315bbdf not found: ID does not exist"
Sep 14 05:02:07.659679 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.659646    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca\") pod \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") "
Sep 14 05:02:07.659854 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.659727    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs\") pod \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") "
Sep 14 05:02:07.659854 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.659833    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") pod \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") "
Sep 14 05:02:07.660013 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.659859    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfvq2\" (UniqueName: \"kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2\") pod \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\" (UID: \"c114b06b-955d-4a49-bbd3-5b559d11ca9b\") "
Sep 14 05:02:07.662435 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.660852    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c114b06b-955d-4a49-bbd3-5b559d11ca9b" (UID: "c114b06b-955d-4a49-bbd3-5b559d11ca9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:02:07.664473 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.664441    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2" (OuterVolumeSpecName: "kube-api-access-dfvq2") pod "c114b06b-955d-4a49-bbd3-5b559d11ca9b" (UID: "c114b06b-955d-4a49-bbd3-5b559d11ca9b"). InnerVolumeSpecName "kube-api-access-dfvq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:02:07.665648 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.665624    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c114b06b-955d-4a49-bbd3-5b559d11ca9b" (UID: "c114b06b-955d-4a49-bbd3-5b559d11ca9b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:02:07.665828 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.665810    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c114b06b-955d-4a49-bbd3-5b559d11ca9b" (UID: "c114b06b-955d-4a49-bbd3-5b559d11ca9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:02:07.680519 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.680381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:07.760872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.760833    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c114b06b-955d-4a49-bbd3-5b559d11ca9b-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:07.760872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.760869    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:07.760872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.760879    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c114b06b-955d-4a49-bbd3-5b559d11ca9b-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:07.761173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.760887    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfvq2\" (UniqueName: \"kubernetes.io/projected/c114b06b-955d-4a49-bbd3-5b559d11ca9b-kube-api-access-dfvq2\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:07.906581 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.906531    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:02:07.908678 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:07.908647    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-654e8bde-64c97f44c6-pjt65"]
Sep 14 05:02:08.093289 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.093240    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:08.192761 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.192673    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:08.195701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.195672    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" path="/var/lib/kubelet/pods/c114b06b-955d-4a49-bbd3-5b559d11ca9b/volumes"
Sep 14 05:02:08.293728 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.293691    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:08.499549 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.499516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:08.679806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:08.679769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:09.039785 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:09.039752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:09.382263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:09.382160    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:10.166650 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:10.166611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:10.577208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:10.577181    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:02:10.962131 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:10.962058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:11.369704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:11.369575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:11.472490 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:11.472445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:11.570286 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:11.570247    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:11.886527 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:11.886492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:12.635799 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.635769    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:12.636251 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.636235    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerName="maas-api"
Sep 14 05:02:12.636324 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.636254    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerName="maas-api"
Sep 14 05:02:12.636366 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.636336    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c114b06b-955d-4a49-bbd3-5b559d11ca9b" containerName="maas-api"
Sep 14 05:02:12.639555 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.639536    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.641907 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.641883    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-60e333a9-serving-cert\""
Sep 14 05:02:12.650257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.650232    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:12.724679 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.724634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:12.816152 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.816098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.816367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.816216    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.816367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.816271    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.816367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.816305    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47nnd\" (UniqueName: \"kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.825530 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.825496    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:12.917806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.917717    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.917806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.917775    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.917806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.917803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47nnd\" (UniqueName: \"kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.918099 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.917848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.918099 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:12.917895    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-serving-cert: secret "maas-api-e2e-migrate-60e333a9-serving-cert" not found
Sep 14 05:02:12.918099 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:12.918014    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls podName:d1063c8e-52ee-43d7-8a87-0da27dd65169 nodeName:}" failed. No retries permitted until 2026-09-14 05:02:13.417992417 +0000 UTC m=+927.779944858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls") pod "maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" (UID: "d1063c8e-52ee-43d7-8a87-0da27dd65169") : secret "maas-api-e2e-migrate-60e333a9-serving-cert" not found
Sep 14 05:02:12.918583 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.918556    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.920533 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.920507    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:12.927078 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:12.927058    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47nnd\" (UniqueName: \"kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:13.288106 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.288068    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:13.391891 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.391853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:13.422233 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.422197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:13.424836 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.424807    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") pod \"maas-api-e2e-migrate-60e333a9-5964976c48-dxllz\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:13.479855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.479815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:13.551336 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.551244    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:13.775312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.775281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:13.840493 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:13.840460    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:13.842217 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:02:13.842185    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1063c8e_52ee_43d7_8a87_0da27dd65169.slice/crio-5ddbb4147de74d2e2327dca7575205e6c0a3750efc3442fa4aa7b2c200c241cf WatchSource:0}: Error finding container 5ddbb4147de74d2e2327dca7575205e6c0a3750efc3442fa4aa7b2c200c241cf: Status 404 returned error can't find the container with id 5ddbb4147de74d2e2327dca7575205e6c0a3750efc3442fa4aa7b2c200c241cf
Sep 14 05:02:14.629339 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.629298    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" event={"ID":"d1063c8e-52ee-43d7-8a87-0da27dd65169","Type":"ContainerStarted","Data":"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"}
Sep 14 05:02:14.629535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.629452    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:14.629535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.629478    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" event={"ID":"d1063c8e-52ee-43d7-8a87-0da27dd65169","Type":"ContainerStarted","Data":"5ddbb4147de74d2e2327dca7575205e6c0a3750efc3442fa4aa7b2c200c241cf"}
Sep 14 05:02:14.648691 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.648539    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" podStartSLOduration=2.308992853 podStartE2EDuration="2.648513766s" totalImagesPullingTime="339.520913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:02:14.645664711 +0000 UTC m=+929.007617153" watchObservedRunningTime="2026-09-14 05:02:14.648513766 +0000 UTC m=+929.010466212"
Sep 14 05:02:14.670668 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.670631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:14.946405 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:14.946317    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:15.658485 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:15.658445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:16.092147 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:16.092111    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:16.367414 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:16.367325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:17.240338 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:17.240298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:18.058129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:18.058095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:18.760343 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:18.760306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:19.203556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:19.203519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:19.473462 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:19.473348    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:20.280477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:20.277773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:20.641821 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:20.641736    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:23.294190 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:23.294153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:33.770063 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:33.770024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:35.268215 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:35.268175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:35.947356 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:35.947307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:36.788096 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:36.788057    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:37.546010 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:37.545970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:38.253580 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:38.253538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:38.952575 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:38.952543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:39.666476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:39.666428    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:40.354077 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:40.354045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:41.151052 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:41.151006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:42.593567 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:42.593522    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.206049 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.206018    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:43.206651 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.206617    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" podUID="d1063c8e-52ee-43d7-8a87-0da27dd65169" containerName="maas-api" containerID="cri-o://ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a" gracePeriod=30
Sep 14 05:02:43.232844 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.232799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.305390 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.305309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.361599 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.361568    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.431169 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.431125    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.482771 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.482741    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:43.488766 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.488734    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.582035 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.581995    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.622057 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622027    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca\") pod \"d1063c8e-52ee-43d7-8a87-0da27dd65169\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") "
Sep 14 05:02:43.622524 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622137    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs\") pod \"d1063c8e-52ee-43d7-8a87-0da27dd65169\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") "
Sep 14 05:02:43.622524 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622201    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") pod \"d1063c8e-52ee-43d7-8a87-0da27dd65169\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") "
Sep 14 05:02:43.622524 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622268    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47nnd\" (UniqueName: \"kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd\") pod \"d1063c8e-52ee-43d7-8a87-0da27dd65169\" (UID: \"d1063c8e-52ee-43d7-8a87-0da27dd65169\") "
Sep 14 05:02:43.622524 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622405    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1063c8e-52ee-43d7-8a87-0da27dd65169" (UID: "d1063c8e-52ee-43d7-8a87-0da27dd65169"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:02:43.622741 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.622630    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1063c8e-52ee-43d7-8a87-0da27dd65169-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:43.624434 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.624402    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd" (OuterVolumeSpecName: "kube-api-access-47nnd") pod "d1063c8e-52ee-43d7-8a87-0da27dd65169" (UID: "d1063c8e-52ee-43d7-8a87-0da27dd65169"). InnerVolumeSpecName "kube-api-access-47nnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:02:43.624554 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.624447    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d1063c8e-52ee-43d7-8a87-0da27dd65169" (UID: "d1063c8e-52ee-43d7-8a87-0da27dd65169"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:02:43.624554 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.624542    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1063c8e-52ee-43d7-8a87-0da27dd65169" (UID: "d1063c8e-52ee-43d7-8a87-0da27dd65169"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:02:43.724056 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.723959    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:43.724056 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.724001    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1063c8e-52ee-43d7-8a87-0da27dd65169-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:43.724056 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.724015    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47nnd\" (UniqueName: \"kubernetes.io/projected/d1063c8e-52ee-43d7-8a87-0da27dd65169-kube-api-access-47nnd\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:02:43.772245 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.772211    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:43.800029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.799988    2571 generic.go:356] "Generic (PLEG): container finished" podID="d1063c8e-52ee-43d7-8a87-0da27dd65169" containerID="ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a" exitCode=0
Sep 14 05:02:43.800237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.800039    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"
Sep 14 05:02:43.800237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.800093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" event={"ID":"d1063c8e-52ee-43d7-8a87-0da27dd65169","Type":"ContainerDied","Data":"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"}
Sep 14 05:02:43.800237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.800175    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz" event={"ID":"d1063c8e-52ee-43d7-8a87-0da27dd65169","Type":"ContainerDied","Data":"5ddbb4147de74d2e2327dca7575205e6c0a3750efc3442fa4aa7b2c200c241cf"}
Sep 14 05:02:43.800237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.800199    2571 scope.go:117] "RemoveContainer" containerID="ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"
Sep 14 05:02:43.810739 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.810720    2571 scope.go:117] "RemoveContainer" containerID="ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"
Sep 14 05:02:43.811128 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:02:43.811107    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a\": container with ID starting with ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a not found: ID does not exist" containerID="ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"
Sep 14 05:02:43.811214 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.811137    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a"} err="failed to get container status \"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a\": rpc error: code = NotFound desc = could not find container \"ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a\": container with ID starting with ea02c2c9d708c0dbe2c37020d257213b0dd9f5138e233918ce1ff99571972a1a not found: ID does not exist"
Sep 14 05:02:43.822551 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.822514    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:43.825904 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:43.825874    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-60e333a9-5964976c48-dxllz"]
Sep 14 05:02:44.073283 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:44.073154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:44.194875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:44.194830    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1063c8e-52ee-43d7-8a87-0da27dd65169" path="/var/lib/kubelet/pods/d1063c8e-52ee-43d7-8a87-0da27dd65169/volumes"
Sep 14 05:02:44.342579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:44.342475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:44.654579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:44.654495    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:44.754857 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:44.754828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:45.372860 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:45.372820    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:46.315800 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:46.315767    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:46.963996 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:46.963956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:47.066857 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:47.066826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:47.225690 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:47.225601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:47.947716 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:47.947679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:48.736547 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.736512    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:02:48.737110 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.737090    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1063c8e-52ee-43d7-8a87-0da27dd65169" containerName="maas-api"
Sep 14 05:02:48.737110 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.737109    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1063c8e-52ee-43d7-8a87-0da27dd65169" containerName="maas-api"
Sep 14 05:02:48.737304 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.737205    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1063c8e-52ee-43d7-8a87-0da27dd65169" containerName="maas-api"
Sep 14 05:02:48.742024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.742003    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.744582 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.744559    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8f579f00-serving-cert\""
Sep 14 05:02:48.751643 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.751618    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:02:48.839433 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.839400    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:48.878210 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.878171    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.878411 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.878249    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m769f\" (UniqueName: \"kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.878411 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.878364    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.878502 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.878414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.940388 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.940357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:48.979443 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.979383    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m769f\" (UniqueName: \"kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.979825 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.979465    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.979825 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.979495    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.979825 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.979572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.980332 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.980301    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.982175 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.982156    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.982257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.982213    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:48.987351 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:48.987290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m769f\" (UniqueName: \"kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f\") pod \"maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:49.054216 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.054181    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:49.337527 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.337364    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:02:49.340406 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:02:49.340377    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a0afb2_998c_43a4_a997_f4432eeb704c.slice/crio-f8f163b160da8f7bfa84c5315f8be0eedaaab716cc0afd01ceed50d1834e18f1 WatchSource:0}: Error finding container f8f163b160da8f7bfa84c5315f8be0eedaaab716cc0afd01ceed50d1834e18f1: Status 404 returned error can't find the container with id f8f163b160da8f7bfa84c5315f8be0eedaaab716cc0afd01ceed50d1834e18f1
Sep 14 05:02:49.341922 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.341901    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 05:02:49.411416 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.411382    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:49.514919 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.514863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:49.837945 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.837883    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" event={"ID":"30a0afb2-998c-43a4-a997-f4432eeb704c","Type":"ContainerStarted","Data":"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"}
Sep 14 05:02:49.838141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.838095    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:02:49.838141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.838120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" event={"ID":"30a0afb2-998c-43a4-a997-f4432eeb704c","Type":"ContainerStarted","Data":"f8f163b160da8f7bfa84c5315f8be0eedaaab716cc0afd01ceed50d1834e18f1"}
Sep 14 05:02:49.876699 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.876646    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" podStartSLOduration=1.497642938 podStartE2EDuration="1.876627354s" totalImagesPullingTime="378.984416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:02:49.875051648 +0000 UTC m=+964.237004094" watchObservedRunningTime="2026-09-14 05:02:49.876627354 +0000 UTC m=+964.238579797"
Sep 14 05:02:49.905274 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:49.905232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:50.660103 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:50.660064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:51.476486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:51.476448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:52.152640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:52.152607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:52.871363 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:52.871319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:02:55.852832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:02:55.852805    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:03:06.928260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:06.928175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:10.960443 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:10.960398    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:11.768790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:11.768754    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:12.466632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:12.466595    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:13.286741 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:13.286698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:14.066092 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:14.066059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:14.752660 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:14.752619    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:15.624440 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:15.624373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:16.685582 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:16.685523    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:17.521175 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:17.521108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:18.255675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:18.255643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:28.346545 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:28.346506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:29.001203 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.001168    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:03:29.001441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.001411    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" podUID="30a0afb2-998c-43a4-a997-f4432eeb704c" containerName="maas-api" containerID="cri-o://c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c" gracePeriod=30
Sep 14 05:03:29.034306 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.034254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:29.094519 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.094484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:29.198496 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.198460    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:29.252454 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.252358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:29.268728 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.268693    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:03:29.398377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.398339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca\") pod \"30a0afb2-998c-43a4-a997-f4432eeb704c\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") "
Sep 14 05:03:29.398879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.398424    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m769f\" (UniqueName: \"kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f\") pod \"30a0afb2-998c-43a4-a997-f4432eeb704c\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") "
Sep 14 05:03:29.398879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.398618    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs\") pod \"30a0afb2-998c-43a4-a997-f4432eeb704c\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") "
Sep 14 05:03:29.398879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.398714    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls\") pod \"30a0afb2-998c-43a4-a997-f4432eeb704c\" (UID: \"30a0afb2-998c-43a4-a997-f4432eeb704c\") "
Sep 14 05:03:29.398879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.398848    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30a0afb2-998c-43a4-a997-f4432eeb704c" (UID: "30a0afb2-998c-43a4-a997-f4432eeb704c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:03:29.399244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.399220    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30a0afb2-998c-43a4-a997-f4432eeb704c-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:03:29.401430 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.401347    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30a0afb2-998c-43a4-a997-f4432eeb704c" (UID: "30a0afb2-998c-43a4-a997-f4432eeb704c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:03:29.401573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.401549    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30a0afb2-998c-43a4-a997-f4432eeb704c" (UID: "30a0afb2-998c-43a4-a997-f4432eeb704c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:03:29.401800 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.401763    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f" (OuterVolumeSpecName: "kube-api-access-m769f") pod "30a0afb2-998c-43a4-a997-f4432eeb704c" (UID: "30a0afb2-998c-43a4-a997-f4432eeb704c"). InnerVolumeSpecName "kube-api-access-m769f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:03:29.500901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.500863    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m769f\" (UniqueName: \"kubernetes.io/projected/30a0afb2-998c-43a4-a997-f4432eeb704c-kube-api-access-m769f\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:03:29.500901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.500897    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:03:29.500901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.500906    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30a0afb2-998c-43a4-a997-f4432eeb704c-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:03:29.678849 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:29.678700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:30.088605 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.088571    2571 generic.go:356] "Generic (PLEG): container finished" podID="30a0afb2-998c-43a4-a997-f4432eeb704c" containerID="c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c" exitCode=0
Sep 14 05:03:30.088790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.088635    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"
Sep 14 05:03:30.088790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.088652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" event={"ID":"30a0afb2-998c-43a4-a997-f4432eeb704c","Type":"ContainerDied","Data":"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"}
Sep 14 05:03:30.088790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.088712    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn" event={"ID":"30a0afb2-998c-43a4-a997-f4432eeb704c","Type":"ContainerDied","Data":"f8f163b160da8f7bfa84c5315f8be0eedaaab716cc0afd01ceed50d1834e18f1"}
Sep 14 05:03:30.088790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.088741    2571 scope.go:117] "RemoveContainer" containerID="c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"
Sep 14 05:03:30.102579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.102551    2571 scope.go:117] "RemoveContainer" containerID="c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"
Sep 14 05:03:30.103152 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:03:30.103118    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c\": container with ID starting with c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c not found: ID does not exist" containerID="c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"
Sep 14 05:03:30.103284 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.103162    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c"} err="failed to get container status \"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c\": rpc error: code = NotFound desc = could not find container \"c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c\": container with ID starting with c7acca5a0daf1db080eadf7888319bd6bb8aac04369434b554931598fad4919c not found: ID does not exist"
Sep 14 05:03:30.114198 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.114165    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:03:30.119002 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.118967    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8f579f00-56f49ff8b-9t4qn"]
Sep 14 05:03:30.195439 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.195400    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a0afb2-998c-43a4-a997-f4432eeb704c" path="/var/lib/kubelet/pods/30a0afb2-998c-43a4-a997-f4432eeb704c/volumes"
Sep 14 05:03:30.400837 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.400753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:30.464171 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.464131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:30.941580 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:30.941551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:31.256233 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:31.256197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:31.356182 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:31.356144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:31.621770 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:31.621686    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:32.074689 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:32.074643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:32.875499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:32.875461    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:33.611340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:33.611304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:33.718385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:33.718351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:33.819231 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:33.819187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:34.459609 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:34.459564    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:35.227509 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:35.227473    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:36.094576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:36.094541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:37.050727 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:37.050690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:37.871531 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:37.871493    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:38.537202 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.537168    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:03:38.538004 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.537978    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a0afb2-998c-43a4-a997-f4432eeb704c" containerName="maas-api"
Sep 14 05:03:38.538004 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.538002    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a0afb2-998c-43a4-a997-f4432eeb704c" containerName="maas-api"
Sep 14 05:03:38.538240 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.538114    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a0afb2-998c-43a4-a997-f4432eeb704c" containerName="maas-api"
Sep 14 05:03:38.542717 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.542694    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.545343 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.545318    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2a2a3973-serving-cert\""
Sep 14 05:03:38.550359 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.550333    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:03:38.566203 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.566152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:38.584762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.584726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxc6\" (UniqueName: \"kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.584971 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.584769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.584971 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.584790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.584971 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.584845    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.667065 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.667024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:38.686465 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.686423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxc6\" (UniqueName: \"kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.686640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.686474    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.686640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.686493    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.686640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.686520    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.687218 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.687193    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.689180 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.689154    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.689180 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.689168    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.695382 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.695356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxc6\" (UniqueName: \"kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6\") pod \"maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:38.858057 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:38.857965    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:39.039733 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:39.039704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:39.136667 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:39.136300    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:03:39.139511 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:03:39.139464    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ef7099_4c21_44a5_a06c_30fdea21fb39.slice/crio-5bd1177b66e0eda3b7473f5f34319f54738d6ac530742dd09500089e19ec0d28 WatchSource:0}: Error finding container 5bd1177b66e0eda3b7473f5f34319f54738d6ac530742dd09500089e19ec0d28: Status 404 returned error can't find the container with id 5bd1177b66e0eda3b7473f5f34319f54738d6ac530742dd09500089e19ec0d28
Sep 14 05:03:39.140860 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:39.140831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:40.149658 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.149623    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" event={"ID":"a2ef7099-4c21-44a5-a06c-30fdea21fb39","Type":"ContainerStarted","Data":"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"}
Sep 14 05:03:40.150138 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.149684    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" event={"ID":"a2ef7099-4c21-44a5-a06c-30fdea21fb39","Type":"ContainerStarted","Data":"5bd1177b66e0eda3b7473f5f34319f54738d6ac530742dd09500089e19ec0d28"}
Sep 14 05:03:40.150138 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.149712    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:40.170554 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.170494    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" podStartSLOduration=1.774808299 podStartE2EDuration="2.17047531s" totalImagesPullingTime="395.667011ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:03:40.168784528 +0000 UTC m=+1014.530736972" watchObservedRunningTime="2026-09-14 05:03:40.17047531 +0000 UTC m=+1014.532427753"
Sep 14 05:03:40.318859 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.318821    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:40.416947 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:40.416828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:03:46.160951 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:46.160904    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:03:58.051532 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:03:58.051487    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:00.646237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:00.646193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:01.497673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:01.497637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:03.519424 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:03.519378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:03.619759 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:03.619718    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:04.338728 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:04.338695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:05.147594 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:05.147560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:05.867861 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:05.867829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:06.581123 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:06.581088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:07.340455 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:07.340415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:08.060977 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:08.060922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:08.759372 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:08.759342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:15.130998 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.130964    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:15.689716 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.689670    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:04:15.689988 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.689921    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" podUID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" containerName="maas-api" containerID="cri-o://e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa" gracePeriod=30
Sep 14 05:04:15.717917 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.717876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:15.779067 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.779016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:15.903402 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.903361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:15.950863 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.950793    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:04:15.964554 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:15.964524    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:16.064535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.064494    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs\") pod \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") "
Sep 14 05:04:16.064700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.064599    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls\") pod \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") "
Sep 14 05:04:16.064748 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.064696    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbxc6\" (UniqueName: \"kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6\") pod \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") "
Sep 14 05:04:16.064800 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.064744    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca\") pod \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\" (UID: \"a2ef7099-4c21-44a5-a06c-30fdea21fb39\") "
Sep 14 05:04:16.065230 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.065195    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2ef7099-4c21-44a5-a06c-30fdea21fb39" (UID: "a2ef7099-4c21-44a5-a06c-30fdea21fb39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:04:16.067124 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.067092    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2ef7099-4c21-44a5-a06c-30fdea21fb39" (UID: "a2ef7099-4c21-44a5-a06c-30fdea21fb39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:04:16.067124 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.067101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2ef7099-4c21-44a5-a06c-30fdea21fb39" (UID: "a2ef7099-4c21-44a5-a06c-30fdea21fb39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:04:16.067889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.067855    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6" (OuterVolumeSpecName: "kube-api-access-mbxc6") pod "a2ef7099-4c21-44a5-a06c-30fdea21fb39" (UID: "a2ef7099-4c21-44a5-a06c-30fdea21fb39"). InnerVolumeSpecName "kube-api-access-mbxc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:04:16.130323 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.130288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:16.165846 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.165810    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:04:16.165846 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.165839    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2ef7099-4c21-44a5-a06c-30fdea21fb39-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:04:16.165846 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.165849    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbxc6\" (UniqueName: \"kubernetes.io/projected/a2ef7099-4c21-44a5-a06c-30fdea21fb39-kube-api-access-mbxc6\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:04:16.166334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.165858    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ef7099-4c21-44a5-a06c-30fdea21fb39-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:04:16.355874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.355836    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" containerID="e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa" exitCode=0
Sep 14 05:04:16.355874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.355881    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"
Sep 14 05:04:16.356163 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.355896    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" event={"ID":"a2ef7099-4c21-44a5-a06c-30fdea21fb39","Type":"ContainerDied","Data":"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"}
Sep 14 05:04:16.356163 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.355978    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk" event={"ID":"a2ef7099-4c21-44a5-a06c-30fdea21fb39","Type":"ContainerDied","Data":"5bd1177b66e0eda3b7473f5f34319f54738d6ac530742dd09500089e19ec0d28"}
Sep 14 05:04:16.356163 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.356003    2571 scope.go:117] "RemoveContainer" containerID="e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"
Sep 14 05:04:16.365731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.365714    2571 scope.go:117] "RemoveContainer" containerID="e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"
Sep 14 05:04:16.366035 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:04:16.366014    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa\": container with ID starting with e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa not found: ID does not exist" containerID="e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"
Sep 14 05:04:16.366095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.366044    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa"} err="failed to get container status \"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa\": rpc error: code = NotFound desc = could not find container \"e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa\": container with ID starting with e95abbb1050894bc8afdfd1cffee7d85541874a3ad213a8860ce2ef360df3faa not found: ID does not exist"
Sep 14 05:04:16.375125 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.375092    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:04:16.378914 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.378889    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a2a3973-bd99954b6-j9htk"]
Sep 14 05:04:16.464329 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.464243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:16.565345 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.565306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:16.847319 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.847222    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:16.951958 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:16.951911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:17.047650 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:17.047609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:17.376925 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:17.376883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:18.195160 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:18.195123    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" path="/var/lib/kubelet/pods/a2ef7099-4c21-44a5-a06c-30fdea21fb39/volumes"
Sep 14 05:04:18.217029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:18.216989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:18.618778 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:18.618743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:18.974304 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:18.974273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:19.770780 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:19.770746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:20.458089 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:20.458052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:25.194811 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:25.194777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:25.339690 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:25.339656    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:25.439130 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:25.439093    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:25.539882 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:25.539835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:25.910435 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:25.910355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:30.091159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:30.091130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:32.933031 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:32.932988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:50.186988 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:50.186950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:51.279572 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:51.279535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:51.874016 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:51.873923    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:52.090203 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:52.090168    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:52.823728 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:52.823697    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:53.544822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:53.544786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:53.781044 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:53.781008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:54.256525 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:54.256491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:54.991489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:54.991456    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:57.591012 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:57.590969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:57.649090 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:57.649055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:57.732740 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:57.732699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:04:57.832821 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:04:57.832779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:00.206913 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:00.206874    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:00.726328 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:00.726282    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.381453 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.380953    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:05:01.381453 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.381234    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" podUID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" containerName="maas-api" containerID="cri-o://652d5c8a9326d0fe4ad45d95f8b2bf442a5c71fcad7c73213b73eeaa18f9ccd9" gracePeriod=30
Sep 14 05:05:01.408679 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.408651    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.472208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.472166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.596493 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.596421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.616105 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.616066    2571 generic.go:356] "Generic (PLEG): container finished" podID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" containerID="652d5c8a9326d0fe4ad45d95f8b2bf442a5c71fcad7c73213b73eeaa18f9ccd9" exitCode=0
Sep 14 05:05:01.616331 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.616176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" event={"ID":"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614","Type":"ContainerDied","Data":"652d5c8a9326d0fe4ad45d95f8b2bf442a5c71fcad7c73213b73eeaa18f9ccd9"}
Sep 14 05:05:01.647496 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.647454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.660068 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.660045    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:05:01.723673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.723643    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca\") pod \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") "
Sep 14 05:05:01.723870 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.723767    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") pod \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") "
Sep 14 05:05:01.723870 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.723809    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8679l\" (UniqueName: \"kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l\") pod \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") "
Sep 14 05:05:01.723870 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.723854    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs\") pod \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\" (UID: \"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614\") "
Sep 14 05:05:01.724070 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.724041    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" (UID: "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:05:01.724184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.724167    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:01.725996 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.725967    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l" (OuterVolumeSpecName: "kube-api-access-8679l") pod "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" (UID: "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614"). InnerVolumeSpecName "kube-api-access-8679l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:05:01.726134 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.726021    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" (UID: "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:05:01.726134 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.726068    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" (UID: "e9bf71b6-5da7-4b39-9c3f-2cad5f01d614"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:05:01.732570 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.732541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:01.825522 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.825483    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:01.825522 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.825526    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8679l\" (UniqueName: \"kubernetes.io/projected/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-kube-api-access-8679l\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:01.825726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.825541    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:01.924113 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:01.924010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:02.190252 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.190163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:02.624307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.624282    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"
Sep 14 05:05:02.624767 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.624317    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc" event={"ID":"e9bf71b6-5da7-4b39-9c3f-2cad5f01d614","Type":"ContainerDied","Data":"d6adbb420202676f812cc6f363ed35520aef8b360108d4d62bcd019faf2ea8c2"}
Sep 14 05:05:02.624767 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.624381    2571 scope.go:117] "RemoveContainer" containerID="652d5c8a9326d0fe4ad45d95f8b2bf442a5c71fcad7c73213b73eeaa18f9ccd9"
Sep 14 05:05:02.641725 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.641698    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:05:02.643889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.643852    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-f14475-756c8b7bc9-mk7tc"]
Sep 14 05:05:02.768961 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:02.768908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:03.461911 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:03.461876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:04.149884 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:04.149852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:04.196031 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:04.195994    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" path="/var/lib/kubelet/pods/e9bf71b6-5da7-4b39-9c3f-2cad5f01d614/volumes"
Sep 14 05:05:04.963419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:04.963384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:05.304052 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:05.303956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:05.680652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:05.680538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:07.226256 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:07.226220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:10.570244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:10.570194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.129743 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.129708    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:05:11.129994 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.129965    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" podUID="db618531-409a-4734-976b-a2576462af31" containerName="maas-api" containerID="cri-o://6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6" gracePeriod=30
Sep 14 05:05:11.145681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.145642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.188311 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.188110    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:05:11.189067 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.188604    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" podUID="da6b144e-a407-430a-9d77-9632e193c663" containerName="payload-pre-processing" containerID="cri-o://447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf" gracePeriod=30
Sep 14 05:05:11.246524 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.246491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.345424 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.345389    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.410760 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.410701    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.446095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.446065    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:05:11.491910 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.491873    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:05:11.526342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.526313    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca\") pod \"db618531-409a-4734-976b-a2576462af31\" (UID: \"db618531-409a-4734-976b-a2576462af31\") "
Sep 14 05:05:11.526522 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.526368    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") pod \"db618531-409a-4734-976b-a2576462af31\" (UID: \"db618531-409a-4734-976b-a2576462af31\") "
Sep 14 05:05:11.526522 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.526458    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs\") pod \"db618531-409a-4734-976b-a2576462af31\" (UID: \"db618531-409a-4734-976b-a2576462af31\") "
Sep 14 05:05:11.526633 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.526524    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87gtw\" (UniqueName: \"kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw\") pod \"db618531-409a-4734-976b-a2576462af31\" (UID: \"db618531-409a-4734-976b-a2576462af31\") "
Sep 14 05:05:11.526731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.526703    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "db618531-409a-4734-976b-a2576462af31" (UID: "db618531-409a-4734-976b-a2576462af31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:05:11.528871 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.528840    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "db618531-409a-4734-976b-a2576462af31" (UID: "db618531-409a-4734-976b-a2576462af31"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:05:11.528871 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.528864    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "db618531-409a-4734-976b-a2576462af31" (UID: "db618531-409a-4734-976b-a2576462af31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:05:11.529113 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.528912    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw" (OuterVolumeSpecName: "kube-api-access-87gtw") pod "db618531-409a-4734-976b-a2576462af31" (UID: "db618531-409a-4734-976b-a2576462af31"). InnerVolumeSpecName "kube-api-access-87gtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:05:11.627423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627385    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwblk\" (UniqueName: \"kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk\") pod \"da6b144e-a407-430a-9d77-9632e193c663\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") "
Sep 14 05:05:11.627423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627422    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume\") pod \"da6b144e-a407-430a-9d77-9632e193c663\" (UID: \"da6b144e-a407-430a-9d77-9632e193c663\") "
Sep 14 05:05:11.627855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627690    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.627855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627706    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87gtw\" (UniqueName: \"kubernetes.io/projected/db618531-409a-4734-976b-a2576462af31-kube-api-access-87gtw\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.627855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627721    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db618531-409a-4734-976b-a2576462af31-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.627855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627732    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db618531-409a-4734-976b-a2576462af31-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.627855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.627836    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da6b144e-a407-430a-9d77-9632e193c663" (UID: "da6b144e-a407-430a-9d77-9632e193c663"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:05:11.629717 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.629692    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk" (OuterVolumeSpecName: "kube-api-access-lwblk") pod "da6b144e-a407-430a-9d77-9632e193c663" (UID: "da6b144e-a407-430a-9d77-9632e193c663"). InnerVolumeSpecName "kube-api-access-lwblk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:05:11.680640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.680536    2571 generic.go:356] "Generic (PLEG): container finished" podID="db618531-409a-4734-976b-a2576462af31" containerID="6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6" exitCode=0
Sep 14 05:05:11.680640 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.680603    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"
Sep 14 05:05:11.680886 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.680632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" event={"ID":"db618531-409a-4734-976b-a2576462af31","Type":"ContainerDied","Data":"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"}
Sep 14 05:05:11.680886 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.680702    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj" event={"ID":"db618531-409a-4734-976b-a2576462af31","Type":"ContainerDied","Data":"fe479baea4d0afcb2060dd213a10f154d799ef4ee0466b9a3b77210bbd349fb2"}
Sep 14 05:05:11.680886 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.680732    2571 scope.go:117] "RemoveContainer" containerID="6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"
Sep 14 05:05:11.682455 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.682414    2571 generic.go:356] "Generic (PLEG): container finished" podID="da6b144e-a407-430a-9d77-9632e193c663" containerID="447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf" exitCode=0
Sep 14 05:05:11.682598 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.682463    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"
Sep 14 05:05:11.682598 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.682507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" event={"ID":"da6b144e-a407-430a-9d77-9632e193c663","Type":"ContainerDied","Data":"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"}
Sep 14 05:05:11.682598 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.682548    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6" event={"ID":"da6b144e-a407-430a-9d77-9632e193c663","Type":"ContainerDied","Data":"a52d0500d4f444569e84dfeb48ba7f094a530d1df2462c2d8b1e45fb9ece2363"}
Sep 14 05:05:11.692204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.692179    2571 scope.go:117] "RemoveContainer" containerID="6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"
Sep 14 05:05:11.692589 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:05:11.692569    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6\": container with ID starting with 6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6 not found: ID does not exist" containerID="6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"
Sep 14 05:05:11.692666 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.692597    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6"} err="failed to get container status \"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6\": rpc error: code = NotFound desc = could not find container \"6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6\": container with ID starting with 6c4fefeaf24777a40b34fbaa4a2036e2358190df42e629256943c392020de8a6 not found: ID does not exist"
Sep 14 05:05:11.692666 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.692614    2571 scope.go:117] "RemoveContainer" containerID="447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"
Sep 14 05:05:11.700191 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.700163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:11.704357 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.704330    2571 scope.go:117] "RemoveContainer" containerID="447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"
Sep 14 05:05:11.704761 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:05:11.704741    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf\": container with ID starting with 447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf not found: ID does not exist" containerID="447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"
Sep 14 05:05:11.704829 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.704771    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf"} err="failed to get container status \"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf\": rpc error: code = NotFound desc = could not find container \"447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf\": container with ID starting with 447b9db807eb693d84eacf5397d2810c944134bb9e276c92937cca96cb8c88cf not found: ID does not exist"
Sep 14 05:05:11.706130 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.706111    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:05:11.709766 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.709744    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-c366d3-6958b69bc6-gncbj"]
Sep 14 05:05:11.719843 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.719799    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:05:11.726175 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.726143    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-c366d3-5589b9798-mjbl6"]
Sep 14 05:05:11.729189 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.729165    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwblk\" (UniqueName: \"kubernetes.io/projected/da6b144e-a407-430a-9d77-9632e193c663-kube-api-access-lwblk\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.729305 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.729190    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da6b144e-a407-430a-9d77-9632e193c663-plugins-config-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:11.770220 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:11.770177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:12.195185 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:12.195151    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da6b144e-a407-430a-9d77-9632e193c663" path="/var/lib/kubelet/pods/da6b144e-a407-430a-9d77-9632e193c663/volumes"
Sep 14 05:05:12.195593 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:12.195577    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db618531-409a-4734-976b-a2576462af31" path="/var/lib/kubelet/pods/db618531-409a-4734-976b-a2576462af31/volumes"
Sep 14 05:05:12.433606 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:12.433517    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:13.199278 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:13.199245    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:13.622028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:13.621997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:14.074023 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:14.073986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:15.239412 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:15.239379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:15.295904 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:15.295870    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:16.076650 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:16.076608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:17.656389 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:17.656355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:17.757212 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:17.757173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:17.881742 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:17.881708    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:05:17.881990 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:17.881961    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy" containerID="cri-o://a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038" gracePeriod=30
Sep 14 05:05:17.897775 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:17.897707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:19.546722 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:19.546669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:19.613636 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:19.613602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:19.688504 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:19.688470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:19.786851 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:19.786777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:23.140536 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.140507    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:05:23.241517 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241480    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241533    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241594    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241622    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241649    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241787    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241821    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46h89\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241849    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.241966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.241894    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert\") pod \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\" (UID: \"ece0edeb-0aeb-4a1b-a3dc-0745008115b2\") "
Sep 14 05:05:23.242177 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242002    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:05:23.242177 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242022    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:05:23.242282 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242202    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:05:23.242419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242307    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.242419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:05:23.242419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242335    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-workload-socket\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.242419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242351    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-data\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.242717 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.242474    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:05:23.244108 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.244076    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:05:23.244108 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.244092    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:05:23.244108 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.244075    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 05:05:23.244287 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.244198    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89" (OuterVolumeSpecName: "kube-api-access-46h89") pod "ece0edeb-0aeb-4a1b-a3dc-0745008115b2" (UID: "ece0edeb-0aeb-4a1b-a3dc-0745008115b2"). InnerVolumeSpecName "kube-api-access-46h89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:05:23.343378 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343343    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-credential-socket\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.343378 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343371    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-envoy\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.343378 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343380    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46h89\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-kube-api-access-46h89\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.343613 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343390    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istiod-ca-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.343613 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343399    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-token\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.343613 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.343407    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ece0edeb-0aeb-4a1b-a3dc-0745008115b2-istio-podinfo\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:05:23.751852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.751819    2571 generic.go:356] "Generic (PLEG): container finished" podID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerID="a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038" exitCode=0
Sep 14 05:05:23.752055 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.751869    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"
Sep 14 05:05:23.752055 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.751902    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerDied","Data":"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"}
Sep 14 05:05:23.752055 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.751969    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5" event={"ID":"ece0edeb-0aeb-4a1b-a3dc-0745008115b2","Type":"ContainerDied","Data":"fb9f16647bf9b91126b34d1fe7ea5e2f78bdd87a862eb4b06296262ffb01f211"}
Sep 14 05:05:23.752055 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.751994    2571 scope.go:117] "RemoveContainer" containerID="a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"
Sep 14 05:05:23.761779 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.761759    2571 scope.go:117] "RemoveContainer" containerID="370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"
Sep 14 05:05:23.772226 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.772200    2571 scope.go:117] "RemoveContainer" containerID="a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"
Sep 14 05:05:23.772724 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:05:23.772691    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038\": container with ID starting with a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038 not found: ID does not exist" containerID="a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"
Sep 14 05:05:23.772847 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.772753    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038"} err="failed to get container status \"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038\": rpc error: code = NotFound desc = could not find container \"a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038\": container with ID starting with a28ed227076205737d76c8b9f9fabe13cebc0901ffeb844784e1ce4966e3e038 not found: ID does not exist"
Sep 14 05:05:23.772847 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.772779    2571 scope.go:117] "RemoveContainer" containerID="370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"
Sep 14 05:05:23.773355 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:05:23.773123    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0\": container with ID starting with 370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0 not found: ID does not exist" containerID="370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"
Sep 14 05:05:23.773355 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.773156    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0"} err="failed to get container status \"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0\": rpc error: code = NotFound desc = could not find container \"370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0\": container with ID starting with 370af7984e918792c87cfa99712a88510d9ff92318e7c60539d7c4be7b2654b0 not found: ID does not exist"
Sep 14 05:05:23.774659 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.774638    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:05:23.777859 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:23.777834    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-c366d3-openshift-default-6584d9b774-tv5q5"]
Sep 14 05:05:24.195656 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:24.195623    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" path="/var/lib/kubelet/pods/ece0edeb-0aeb-4a1b-a3dc-0745008115b2/volumes"
Sep 14 05:05:30.235183 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:30.235138    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:31.026480 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:31.026443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:37.046604 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:37.046562    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:37.148562 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:37.148526    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:37.247382 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:37.247341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:37.346343 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:37.346257    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:38.067386 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:38.067352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:38.135395 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:38.135358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:38.236377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:38.236344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:39.813845 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:39.813813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:41.998992 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:41.998954    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:42.498548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:42.498513    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:44.414701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:44.414666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:44.740327 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:44.740291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:45.239248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:45.239199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:45.566996 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:45.566887    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:45.931129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:45.931017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:45.987786 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:45.987750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:46.202730 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:46.202698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:46.652550 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:46.652452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:47.960211 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:47.960170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:48.636265 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:48.636225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:48.747406 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:48.747369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:48.847754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:48.847719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:48.949993 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:48.949857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:49.558916 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:49.558872    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:50.370103 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:50.370064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.134350 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.134314    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:05:51.135006 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.134984    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" containerName="maas-api"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135008    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" containerName="maas-api"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135056    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" containerName="maas-api"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135065    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" containerName="maas-api"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135077    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135086    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135098    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da6b144e-a407-430a-9d77-9632e193c663" containerName="payload-pre-processing"
Sep 14 05:05:51.135129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135106    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="da6b144e-a407-430a-9d77-9632e193c663" containerName="payload-pre-processing"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135132    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db618531-409a-4734-976b-a2576462af31" containerName="maas-api"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135141    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="db618531-409a-4734-976b-a2576462af31" containerName="maas-api"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135152    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135161    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135266    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9bf71b6-5da7-4b39-9c3f-2cad5f01d614" containerName="maas-api"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135278    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135289    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="da6b144e-a407-430a-9d77-9632e193c663" containerName="payload-pre-processing"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135301    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="db618531-409a-4734-976b-a2576462af31" containerName="maas-api"
Sep 14 05:05:51.135476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.135313    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ef7099-4c21-44a5-a06c-30fdea21fb39" containerName="maas-api"
Sep 14 05:05:51.140376 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.140348    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.142994 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.142961    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-9529a1-serving-cert\""
Sep 14 05:05:51.149845 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.149811    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:05:51.162989 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.162953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.212553 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.212517    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.212738 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.212559    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.212738 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.212588    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z6m6\" (UniqueName: \"kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.212821 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.212729    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.263440 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.263399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.314234 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.314187    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.314417 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.314243    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.314417 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.314299    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z6m6\" (UniqueName: \"kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.314563 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.314439    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.315062 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.315022    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.317034 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.316993    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.317146 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.317119    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.323765 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.323726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z6m6\" (UniqueName: \"kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6\") pod \"maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.363346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.363283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.454679 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.454646    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:51.741921 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.741892    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.742385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.742364    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:05:51.744877 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:05:51.744848    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb0fb9c3_ffee_4fe9_aa93_2b0f51902643.slice/crio-6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248 WatchSource:0}: Error finding container 6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248: Status 404 returned error can't find the container with id 6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248
Sep 14 05:05:51.826549 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.826506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:51.906062 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:51.906018    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" event={"ID":"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643","Type":"ContainerStarted","Data":"6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248"}
Sep 14 05:05:52.914193 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:52.914149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" event={"ID":"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643","Type":"ContainerStarted","Data":"ee59d2c6542718428544d18d6ebd3034de457e997ffcf2fe74fdf01d34f5d450"}
Sep 14 05:05:52.914613 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:52.914223    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:52.934382 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:52.934324    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" podStartSLOduration=1.500109937 podStartE2EDuration="1.934305057s" totalImagesPullingTime="434.19512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:05:52.932709264 +0000 UTC m=+1147.294661708" watchObservedRunningTime="2026-09-14 05:05:52.934305057 +0000 UTC m=+1147.296257500"
Sep 14 05:05:53.220454 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:53.220423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:53.542898 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:53.542808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:53.643235 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:53.643201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:53.785492 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:53.785455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:54.118659 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:54.118628    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:54.400268 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:54.400187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:05:58.925806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:58.925773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:05:59.655188 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:05:59.655141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:00.327173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:00.327137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:00.419396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:00.419354    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:00.646293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:00.646204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:01.366205 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:01.366167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:02.403844 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:02.403807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:03.064288 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:03.064257    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:03.437536 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:03.437453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:03.775787 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:03.775747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:04.447838 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.447800    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:04.449204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.449174    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece0edeb-0aeb-4a1b-a3dc-0745008115b2" containerName="istio-proxy"
Sep 14 05:06:04.453161 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.453123    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.455991 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.455964    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4ee4bdd9-serving-cert\""
Sep 14 05:06:04.461575 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.461498    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:04.489825 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.489702    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:04.551135 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.551104    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtfqb\" (UniqueName: \"kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.551398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.551166    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.551398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.551247    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.551398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.551301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.551398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.551364    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:04.652569 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.652527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtfqb\" (UniqueName: \"kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.652762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.652596    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.652762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.652629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.652762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.652662    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.653457 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.653429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.655229 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.655207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.655321 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.655289    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.661862 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.661829    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtfqb\" (UniqueName: \"kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb\") pod \"maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.768250 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.768210    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:04.939183 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:04.939145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:05.041827 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.041804    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:05.044059 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:06:05.044028    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef8176b_5639_4e18_8b9d_70235ca61635.slice/crio-f76453ba81df0556ee16690cb08b02e4ebaee5525100cb9aa8f5d0d5ef772a36 WatchSource:0}: Error finding container f76453ba81df0556ee16690cb08b02e4ebaee5525100cb9aa8f5d0d5ef772a36: Status 404 returned error can't find the container with id f76453ba81df0556ee16690cb08b02e4ebaee5525100cb9aa8f5d0d5ef772a36
Sep 14 05:06:05.169855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.169819    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:05.257219 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.257184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:05.567204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.567170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:05.987998 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.987960    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" event={"ID":"eef8176b-5639-4e18-8b9d-70235ca61635","Type":"ContainerStarted","Data":"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"}
Sep 14 05:06:05.988174 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.988014    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" event={"ID":"eef8176b-5639-4e18-8b9d-70235ca61635","Type":"ContainerStarted","Data":"f76453ba81df0556ee16690cb08b02e4ebaee5525100cb9aa8f5d0d5ef772a36"}
Sep 14 05:06:05.988174 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:05.988078    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:06.005361 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:06.005305    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" podStartSLOduration=1.6929562630000001 podStartE2EDuration="2.005280795s" totalImagesPullingTime="312.324532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:06:06.003792888 +0000 UTC m=+1160.365745332" watchObservedRunningTime="2026-09-14 05:06:06.005280795 +0000 UTC m=+1160.367233238"
Sep 14 05:06:06.255553 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:06.255450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:06.868954 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:06.868891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:06.928458 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:06.928425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:06.986497 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:06.986465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:07.725136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:07.725104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:08.580997 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:08.580962    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:11.999141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:11.999110    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:13.172509 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:13.172474    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:25.548807 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:25.548769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:27.059549 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:27.059516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:27.777179 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:27.777143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:28.396248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:28.396211    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:29.280303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:29.280261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:29.964260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:29.964212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:30.500427 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:30.500393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:31.222406 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:31.222376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:31.319573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:31.319534    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:31.692563 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:31.692481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:37.680314 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:37.680274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:37.748299 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:37.748260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:38.948250 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:38.948217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:39.772914 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:39.772880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:40.487820 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:40.487778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:41.267727 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:41.267695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:42.057905 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:42.057878    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:42.738545 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:42.738506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:46.192048 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:46.192015    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:06:46.195497 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:46.195466    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:06:46.197767 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:46.197742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:06:46.200669 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:46.200649    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:06:48.518339 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.518300    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:48.518879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.518641    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" podUID="eef8176b-5639-4e18-8b9d-70235ca61635" containerName="maas-api" containerID="cri-o://04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450" gracePeriod=30
Sep 14 05:06:48.546696 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.546668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:48.658783 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.658708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:48.735511 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.735464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:48.782428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.782351    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:48.794755 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.794720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:48.892748 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.892698    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls\") pod \"eef8176b-5639-4e18-8b9d-70235ca61635\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") "
Sep 14 05:06:48.892748 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.892745    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtfqb\" (UniqueName: \"kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb\") pod \"eef8176b-5639-4e18-8b9d-70235ca61635\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") "
Sep 14 05:06:48.893065 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.892839    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs\") pod \"eef8176b-5639-4e18-8b9d-70235ca61635\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") "
Sep 14 05:06:48.893065 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.892894    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca\") pod \"eef8176b-5639-4e18-8b9d-70235ca61635\" (UID: \"eef8176b-5639-4e18-8b9d-70235ca61635\") "
Sep 14 05:06:48.893289 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.893248    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eef8176b-5639-4e18-8b9d-70235ca61635" (UID: "eef8176b-5639-4e18-8b9d-70235ca61635"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:06:48.895119 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.895085    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eef8176b-5639-4e18-8b9d-70235ca61635" (UID: "eef8176b-5639-4e18-8b9d-70235ca61635"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:06:48.895119 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.895092    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb" (OuterVolumeSpecName: "kube-api-access-jtfqb") pod "eef8176b-5639-4e18-8b9d-70235ca61635" (UID: "eef8176b-5639-4e18-8b9d-70235ca61635"). InnerVolumeSpecName "kube-api-access-jtfqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:06:48.895270 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.895138    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eef8176b-5639-4e18-8b9d-70235ca61635" (UID: "eef8176b-5639-4e18-8b9d-70235ca61635"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:06:48.994373 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.994328    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:06:48.994373 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.994355    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef8176b-5639-4e18-8b9d-70235ca61635-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:06:48.994373 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.994365    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eef8176b-5639-4e18-8b9d-70235ca61635-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:06:48.994373 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:48.994376    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtfqb\" (UniqueName: \"kubernetes.io/projected/eef8176b-5639-4e18-8b9d-70235ca61635-kube-api-access-jtfqb\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:06:49.132292 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.132178    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:49.224647 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.224610    2571 generic.go:356] "Generic (PLEG): container finished" podID="eef8176b-5639-4e18-8b9d-70235ca61635" containerID="04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450" exitCode=0
Sep 14 05:06:49.224850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.224665    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"
Sep 14 05:06:49.224850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.224701    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" event={"ID":"eef8176b-5639-4e18-8b9d-70235ca61635","Type":"ContainerDied","Data":"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"}
Sep 14 05:06:49.224850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.224760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf" event={"ID":"eef8176b-5639-4e18-8b9d-70235ca61635","Type":"ContainerDied","Data":"f76453ba81df0556ee16690cb08b02e4ebaee5525100cb9aa8f5d0d5ef772a36"}
Sep 14 05:06:49.224850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.224787    2571 scope.go:117] "RemoveContainer" containerID="04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"
Sep 14 05:06:49.231882 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.231846    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:49.234889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.234871    2571 scope.go:117] "RemoveContainer" containerID="04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"
Sep 14 05:06:49.235278 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:06:49.235257    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450\": container with ID starting with 04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450 not found: ID does not exist" containerID="04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"
Sep 14 05:06:49.235326 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.235288    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450"} err="failed to get container status \"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450\": rpc error: code = NotFound desc = could not find container \"04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450\": container with ID starting with 04cbff4eb62276a77484473982ca2791c5244ba808d3e88882f9b9c830cfe450 not found: ID does not exist"
Sep 14 05:06:49.245908 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.245881    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:49.249482 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.249461    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4ee4bdd9-6bc75469cf-5rfdf"]
Sep 14 05:06:49.857649 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:49.857614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:50.194963 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:50.194845    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef8176b-5639-4e18-8b9d-70235ca61635" path="/var/lib/kubelet/pods/eef8176b-5639-4e18-8b9d-70235ca61635/volumes"
Sep 14 05:06:50.665085 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:50.665046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:51.366591 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:51.366556    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:52.032323 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:52.032274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:55.357393 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:55.357358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:55.453870 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:55.453830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:55.645173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:55.645082    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:56.041824 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:56.041788    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:56.142652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:56.142610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:56.986472 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:56.986422    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:57.268987 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:57.268840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:06:57.965154 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:06:57.965094    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:02.295299 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:02.295264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:02.392722 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:02.392689    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:02.493289 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:02.493255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:02.546159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:02.546065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:02.697623 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:02.697591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:04.172115 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:04.172080    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:08.402451 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:08.402416    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:08.503382 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:08.503342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:09.448307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:09.448273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:09.752355 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:09.752317    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:09.860313 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:09.860275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:12.038848 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:12.038811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:12.653152 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:12.653112    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:12.747208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:12.747177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:12.858377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:12.858344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:13.671773 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:13.671698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:14.463719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:14.463682    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:15.244619 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:15.244574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:16.034300 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.034256    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:07:16.034831 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.034812    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef8176b-5639-4e18-8b9d-70235ca61635" containerName="maas-api"
Sep 14 05:07:16.034963 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.034833    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef8176b-5639-4e18-8b9d-70235ca61635" containerName="maas-api"
Sep 14 05:07:16.035040 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.034988    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef8176b-5639-4e18-8b9d-70235ca61635" containerName="maas-api"
Sep 14 05:07:16.038424 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.038400    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.041092 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.041065    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-2eeb50-serving-cert\""
Sep 14 05:07:16.049959 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.049727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:07:16.063810 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.063769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:16.114757 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.114727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:16.162687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.162646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.162875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.162694    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.162875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.162726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2nr7\" (UniqueName: \"kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.162875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.162820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.264270 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.264225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.264719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.264345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.264719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.264371    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.264719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.264386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2nr7\" (UniqueName: \"kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.265303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.265199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.267210 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.267186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.267335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.267186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.272665 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.272633    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2nr7\" (UniqueName: \"kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7\") pod \"maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.277473 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.277449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:16.351792 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.351680    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:16.622182 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.622090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:16.634063 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.634031    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:07:16.636821 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:07:16.636773    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b785fa4_8517_4aa9_8d28_37c32e73e13b.slice/crio-c666a067ed4745006dc1bc2c93dfbcffca756e1b7163d9db94b038063bfbe10e WatchSource:0}: Error finding container c666a067ed4745006dc1bc2c93dfbcffca756e1b7163d9db94b038063bfbe10e: Status 404 returned error can't find the container with id c666a067ed4745006dc1bc2c93dfbcffca756e1b7163d9db94b038063bfbe10e
Sep 14 05:07:16.718211 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:16.718173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:17.388283 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:17.388191    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" event={"ID":"6b785fa4-8517-4aa9-8d28-37c32e73e13b","Type":"ContainerStarted","Data":"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"}
Sep 14 05:07:17.388663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:17.388342    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:17.388663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:17.388391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" event={"ID":"6b785fa4-8517-4aa9-8d28-37c32e73e13b","Type":"ContainerStarted","Data":"c666a067ed4745006dc1bc2c93dfbcffca756e1b7163d9db94b038063bfbe10e"}
Sep 14 05:07:17.405565 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:17.405510    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" podStartSLOduration=0.931293861 podStartE2EDuration="1.405491718s" totalImagesPullingTime="474.197857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:07:17.404078402 +0000 UTC m=+1231.766030846" watchObservedRunningTime="2026-09-14 05:07:17.405491718 +0000 UTC m=+1231.767444160"
Sep 14 05:07:18.051522 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:18.051483    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:22.018818 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:22.018776    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:23.399993 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:23.399962    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:07:28.059008 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:28.058965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:36.997536 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:36.997409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:38.506118 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:38.506084    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:39.139363 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:39.139324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:39.919386 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:39.919353    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:40.702855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:40.702819    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:41.358094 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:41.358056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:42.053913 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:42.053855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:42.924634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:42.924594    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:43.278520 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:43.278483    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:43.637376 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:43.637286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:44.014878 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:44.014842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:46.218642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:46.218603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:46.524924 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:46.524834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:46.805567 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:46.805470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:46.905005 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:46.904968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:47.076262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:47.076170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:47.977841 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:47.977805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:48.648680 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:48.648645    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:49.555143 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:49.555115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:50.113071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.113036    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:50.764307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.764271    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:07:50.769425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.769391    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:50.771956 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.771916    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-a8c46f-serving-cert\""
Sep 14 05:07:50.797265 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.797234    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:07:50.917873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.917832    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:50.918109 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.917893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6gdn\" (UniqueName: \"kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:50.918109 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.918020    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:50.918219 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.918155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:50.925713 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:50.925683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:51.019576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.019486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.019576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.019549    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.019771 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.019590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6gdn\" (UniqueName: \"kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.019771 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.019624    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.020333 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.020261    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.022492 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.022461    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.022492 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.022462    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.025737 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.025708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:51.027539 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.027517    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6gdn\" (UniqueName: \"kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn\") pod \"maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.098309 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.098272    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:51.150857 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.150763    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:51.377450 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.377419    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:07:51.378683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.378657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:51.380992 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:07:51.380961    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d21b65d_529b_4849_ab0f_e47da120bfcf.slice/crio-1ac5587239dcf78d1737cf0ea6b340b82e337c49038bc9c266458ca9c5f487b4 WatchSource:0}: Error finding container 1ac5587239dcf78d1737cf0ea6b340b82e337c49038bc9c266458ca9c5f487b4: Status 404 returned error can't find the container with id 1ac5587239dcf78d1737cf0ea6b340b82e337c49038bc9c266458ca9c5f487b4
Sep 14 05:07:51.382751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.382729    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 05:07:51.479097 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.479034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:07:51.582536 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:51.582493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" event={"ID":"6d21b65d-529b-4849-ab0f-e47da120bfcf","Type":"ContainerStarted","Data":"1ac5587239dcf78d1737cf0ea6b340b82e337c49038bc9c266458ca9c5f487b4"}
Sep 14 05:07:52.589981 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:52.589944    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" event={"ID":"6d21b65d-529b-4849-ab0f-e47da120bfcf","Type":"ContainerStarted","Data":"23efe3aac1c5530103f2d91a3f9a1642369723c300036da6e9b0e9477672c55a"}
Sep 14 05:07:52.590358 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:52.590103    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:07:52.609335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:52.609285    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" podStartSLOduration=2.134233652 podStartE2EDuration="2.609266007s" totalImagesPullingTime="475.032355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:07:52.608405279 +0000 UTC m=+1266.970357723" watchObservedRunningTime="2026-09-14 05:07:52.609266007 +0000 UTC m=+1266.971218451"
Sep 14 05:07:58.601739 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:07:58.601705    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:08:09.827424 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.827379    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:08:09.833065 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.833030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.836806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.836760    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e08402\"/\"kube-root-ca.crt\""
Sep 14 05:08:09.837038 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.836807    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e08402\"/\"e2e-conflict-a-c6882fdf-kserve-self-signed-certs\""
Sep 14 05:08:09.837038 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.836867    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e08402\"/\"default-dockercfg-26l6j\""
Sep 14 05:08:09.837038 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.836875    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e08402\"/\"openshift-service-ca.crt\""
Sep 14 05:08:09.842812 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.842745    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:08:09.905343 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.905548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905362    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.905548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.905548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905475    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6dz\" (UniqueName: \"kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.905548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:09.905716 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:09.905584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006521 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006479    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006533    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006571    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006607    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6dz\" (UniqueName: \"kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006661    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.006916 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.007024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.006916    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.007086 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.007015    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.007086 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.007061    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.009217 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.009195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.009489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.009469    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.015651 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.015626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6dz\" (UniqueName: \"kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz\") pod \"e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") " pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.146238 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.146141    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:10.314587 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.314548    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:10.424952 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.424913    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:08:10.426828 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:10.426800    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00767c2a_9dfa_4141_8c73_691fdc3aedd6.slice/crio-e898f690a475657082708ea8e152e687ead29f9d439eaca776eacae9e462f712 WatchSource:0}: Error finding container e898f690a475657082708ea8e152e687ead29f9d439eaca776eacae9e462f712: Status 404 returned error can't find the container with id e898f690a475657082708ea8e152e687ead29f9d439eaca776eacae9e462f712
Sep 14 05:08:10.697385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.697287    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" event={"ID":"00767c2a-9dfa-4141-8c73-691fdc3aedd6","Type":"ContainerStarted","Data":"d25748f0e2a02a7fa09367b5962585bb3c6e108fb2c3365b6c96457d2f1881f0"}
Sep 14 05:08:10.697385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.697341    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" event={"ID":"00767c2a-9dfa-4141-8c73-691fdc3aedd6","Type":"ContainerStarted","Data":"e898f690a475657082708ea8e152e687ead29f9d439eaca776eacae9e462f712"}
Sep 14 05:08:10.714029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:10.713970    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" podStartSLOduration=1.713954902 podStartE2EDuration="1.713954902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:10.712845058 +0000 UTC m=+1285.074797512" watchObservedRunningTime="2026-09-14 05:08:10.713954902 +0000 UTC m=+1285.075907342"
Sep 14 05:08:12.861443 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:12.861405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:13.587758 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:13.587716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:13.746255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:13.746223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:14.539602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:14.539564    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:15.450409 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:15.450376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:16.097215 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:16.097179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:16.769403 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:16.769363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:16.952291 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:16.952256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:17.772822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.772782    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-576b01"]
Sep 14 05:08:17.777386 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.777360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:17.780226 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.780198    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dcz7p\""
Sep 14 05:08:17.784644 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.784618    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-576b01"]
Sep 14 05:08:17.811220 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.811169    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:17.885901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.885853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzsnh\" (UniqueName: \"kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh\") pod \"test-curl-7533-576b01\" (UID: \"6e5a33b9-dc45-4935-981f-8089cc2cd7d3\") " pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:17.986497 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.986451    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzsnh\" (UniqueName: \"kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh\") pod \"test-curl-7533-576b01\" (UID: \"6e5a33b9-dc45-4935-981f-8089cc2cd7d3\") " pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:17.996976 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:17.996945    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzsnh\" (UniqueName: \"kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh\") pod \"test-curl-7533-576b01\" (UID: \"6e5a33b9-dc45-4935-981f-8089cc2cd7d3\") " pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:18.091790 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:18.091697    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:18.366057 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:18.366029    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-576b01"]
Sep 14 05:08:18.368553 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:18.368512    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e5a33b9_dc45_4935_981f_8089cc2cd7d3.slice/crio-d285095d71d93f8d69ddc1e96f8d2ef730fa780c850551ed5f3962215dae329c WatchSource:0}: Error finding container d285095d71d93f8d69ddc1e96f8d2ef730fa780c850551ed5f3962215dae329c: Status 404 returned error can't find the container with id d285095d71d93f8d69ddc1e96f8d2ef730fa780c850551ed5f3962215dae329c
Sep 14 05:08:18.452277 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:18.452237    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:18.745589 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:18.745543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-576b01" event={"ID":"6e5a33b9-dc45-4935-981f-8089cc2cd7d3","Type":"ContainerStarted","Data":"d285095d71d93f8d69ddc1e96f8d2ef730fa780c850551ed5f3962215dae329c"}
Sep 14 05:08:19.096714 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:19.096627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:20.146795 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:20.146750    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:20.147335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:20.146816    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:20.162324 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:20.162294    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:20.775889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:20.775840    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:08:21.766160 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:21.766115    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-576b01" event={"ID":"6e5a33b9-dc45-4935-981f-8089cc2cd7d3","Type":"ContainerStarted","Data":"f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd"}
Sep 14 05:08:21.781508 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:21.781438    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7533-576b01" podStartSLOduration=2.125108433 podStartE2EDuration="4.781422533s" totalImagesPullingTime="2.6563141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:21.779649274 +0000 UTC m=+1296.141601716" watchObservedRunningTime="2026-09-14 05:08:21.781422533 +0000 UTC m=+1296.143374976"
Sep 14 05:08:22.390706 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.390661    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-576b01"]
Sep 14 05:08:22.393025 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.392992    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-576b01"]
Sep 14 05:08:22.508332 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.508291    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-945b9b"]
Sep 14 05:08:22.508822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.508806    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" containerName="test-curl-7533-576b01"
Sep 14 05:08:22.508868 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.508825    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" containerName="test-curl-7533-576b01"
Sep 14 05:08:22.508912 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.508902    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" containerName="test-curl-7533-576b01"
Sep 14 05:08:22.512400 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.512381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:22.518509 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.518481    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-945b9b"]
Sep 14 05:08:22.633691 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.633652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlpxf\" (UniqueName: \"kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf\") pod \"test-curl-7533-945b9b\" (UID: \"fc1c095b-1f97-444b-8623-2b057d4f38e5\") " pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:22.735169 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.735125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlpxf\" (UniqueName: \"kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf\") pod \"test-curl-7533-945b9b\" (UID: \"fc1c095b-1f97-444b-8623-2b057d4f38e5\") " pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:22.743700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.743674    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlpxf\" (UniqueName: \"kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf\") pod \"test-curl-7533-945b9b\" (UID: \"fc1c095b-1f97-444b-8623-2b057d4f38e5\") " pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:22.773381 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:22.773290    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7533-576b01" container="test-curl-7533-576b01"
Sep 14 05:08:22.823369 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:22.823330    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:23.096405 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.096375    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-945b9b"]
Sep 14 05:08:23.098203 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:23.098165    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc1c095b_1f97_444b_8623_2b057d4f38e5.slice/crio-68c8d8e61bef51ea0e752402274e70879c8e719262b955daabdd764188f797c3 WatchSource:0}: Error finding container 68c8d8e61bef51ea0e752402274e70879c8e719262b955daabdd764188f797c3: Status 404 returned error can't find the container with id 68c8d8e61bef51ea0e752402274e70879c8e719262b955daabdd764188f797c3
Sep 14 05:08:23.781754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.781719    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-945b9b" event={"ID":"fc1c095b-1f97-444b-8623-2b057d4f38e5","Type":"ContainerStarted","Data":"79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f"}
Sep 14 05:08:23.781754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.781764    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-945b9b" event={"ID":"fc1c095b-1f97-444b-8623-2b057d4f38e5","Type":"ContainerStarted","Data":"68c8d8e61bef51ea0e752402274e70879c8e719262b955daabdd764188f797c3"}
Sep 14 05:08:23.782283 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.781820    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7533-576b01" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" containerName="test-curl-7533-576b01" containerID="cri-o://f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd" gracePeriod=2
Sep 14 05:08:23.784087 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:23.784057    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-576b01\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:23.800476 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.800425    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7533-945b9b" podStartSLOduration=1.800405228 podStartE2EDuration="1.800405228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:23.799794286 +0000 UTC m=+1298.161746729" watchObservedRunningTime="2026-09-14 05:08:23.800405228 +0000 UTC m=+1298.162357671"
Sep 14 05:08:23.801839 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:23.801812    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-576b01\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:23.968146 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:23.968110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:24.417220 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.417181    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-945b9b"]
Sep 14 05:08:24.421529 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.421483    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-945b9b"]
Sep 14 05:08:24.518286 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.518247    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:08:24.518519 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.518477    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" podUID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" containerName="maas-api" containerID="cri-o://ee59d2c6542718428544d18d6ebd3034de457e997ffcf2fe74fdf01d34f5d450" gracePeriod=30
Sep 14 05:08:24.539531 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.539500    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-2aa8e5"]
Sep 14 05:08:24.540237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.540178    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" containerName="test-curl-7533-945b9b"
Sep 14 05:08:24.540237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.540203    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" containerName="test-curl-7533-945b9b"
Sep 14 05:08:24.540441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.540312    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" containerName="test-curl-7533-945b9b"
Sep 14 05:08:24.543701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.543674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:24.548084 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.548059    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-2aa8e5"]
Sep 14 05:08:24.548293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.548276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:24.617403 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.617366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:24.654767 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.654724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p8wq\" (UniqueName: \"kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq\") pod \"test-curl-7533-2aa8e5\" (UID: \"8df1316a-9f3a-46fd-b17d-fdf847b0ab64\") " pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:24.718220 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.718189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:24.756289 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.756256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p8wq\" (UniqueName: \"kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq\") pod \"test-curl-7533-2aa8e5\" (UID: \"8df1316a-9f3a-46fd-b17d-fdf847b0ab64\") " pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:24.766797 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.766767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p8wq\" (UniqueName: \"kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq\") pod \"test-curl-7533-2aa8e5\" (UID: \"8df1316a-9f3a-46fd-b17d-fdf847b0ab64\") " pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:24.780413 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.780382    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:24.789110 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.789080    2571 generic.go:356] "Generic (PLEG): container finished" podID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" containerID="ee59d2c6542718428544d18d6ebd3034de457e997ffcf2fe74fdf01d34f5d450" exitCode=0
Sep 14 05:08:24.789463 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.789149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" event={"ID":"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643","Type":"ContainerDied","Data":"ee59d2c6542718428544d18d6ebd3034de457e997ffcf2fe74fdf01d34f5d450"}
Sep 14 05:08:24.789463 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.789203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69" event={"ID":"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643","Type":"ContainerDied","Data":"6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248"}
Sep 14 05:08:24.789463 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.789232    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6226053f3a749fbab40d4807f8648609bd4a668da3ed683029f6affb6cfea248"
Sep 14 05:08:24.789463 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:24.789360    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7533-945b9b" container="test-curl-7533-945b9b"
Sep 14 05:08:24.797071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.797045    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:08:24.857363 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.857334    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs\") pod \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") "
Sep 14 05:08:24.857556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.857382    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls\") pod \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") "
Sep 14 05:08:24.857556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.857442    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z6m6\" (UniqueName: \"kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6\") pod \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") "
Sep 14 05:08:24.857556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.857486    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca\") pod \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\" (UID: \"cb0fb9c3-ffee-4fe9-aa93-2b0f51902643\") "
Sep 14 05:08:24.858366 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.858303    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" (UID: "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:08:24.859850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.859758    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" (UID: "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:08:24.860009 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.859975    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" (UID: "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:08:24.860408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.860372    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6" (OuterVolumeSpecName: "kube-api-access-5z6m6") pod "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" (UID: "cb0fb9c3-ffee-4fe9-aa93-2b0f51902643"). InnerVolumeSpecName "kube-api-access-5z6m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:08:24.892302 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.892270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:24.958683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.958654    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:24.958683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.958684    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:24.958888 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.958697    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z6m6\" (UniqueName: \"kubernetes.io/projected/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-kube-api-access-5z6m6\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:24.958888 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:24.958711    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:25.125203 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.125034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:25.167849 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.167820    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-2aa8e5"]
Sep 14 05:08:25.170651 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:25.170616    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8df1316a_9f3a_46fd_b17d_fdf847b0ab64.slice/crio-559873c9d944d61b7f7fa0ef871a3f110c32e478eebc27f8e1ac9ba2e598db64 WatchSource:0}: Error finding container 559873c9d944d61b7f7fa0ef871a3f110c32e478eebc27f8e1ac9ba2e598db64: Status 404 returned error can't find the container with id 559873c9d944d61b7f7fa0ef871a3f110c32e478eebc27f8e1ac9ba2e598db64
Sep 14 05:08:25.225573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.225543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:25.797145 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.797103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-2aa8e5" event={"ID":"8df1316a-9f3a-46fd-b17d-fdf847b0ab64","Type":"ContainerStarted","Data":"364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc"}
Sep 14 05:08:25.797475 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.797170    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-2aa8e5" event={"ID":"8df1316a-9f3a-46fd-b17d-fdf847b0ab64","Type":"ContainerStarted","Data":"559873c9d944d61b7f7fa0ef871a3f110c32e478eebc27f8e1ac9ba2e598db64"}
Sep 14 05:08:25.797475 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.797122    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"
Sep 14 05:08:25.797475 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.797241    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7533-945b9b" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" containerName="test-curl-7533-945b9b" containerID="cri-o://79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f" gracePeriod=2
Sep 14 05:08:25.806620 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:25.806588    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-945b9b\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:25.821912 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.821867    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7533-2aa8e5" podStartSLOduration=1.821846496 podStartE2EDuration="1.821846496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:25.81949286 +0000 UTC m=+1300.181445304" watchObservedRunningTime="2026-09-14 05:08:25.821846496 +0000 UTC m=+1300.183798941"
Sep 14 05:08:25.822065 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:25.821883    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-945b9b\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:25.855264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.855230    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:25.860681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.860651    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:08:25.865345 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:25.865298    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-945b9b\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:25.866102 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.866082    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-9529a1-66b8d9b788-wjn69"]
Sep 14 05:08:25.982998 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:25.982970    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:25.985343 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:25.985303    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-945b9b\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:26.072517 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.072428    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzsnh\" (UniqueName: \"kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh\") pod \"6e5a33b9-dc45-4935-981f-8089cc2cd7d3\" (UID: \"6e5a33b9-dc45-4935-981f-8089cc2cd7d3\") "
Sep 14 05:08:26.074880 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.074848    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh" (OuterVolumeSpecName: "kube-api-access-nzsnh") pod "6e5a33b9-dc45-4935-981f-8089cc2cd7d3" (UID: "6e5a33b9-dc45-4935-981f-8089cc2cd7d3"). InnerVolumeSpecName "kube-api-access-nzsnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:08:26.174259 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.174215    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzsnh\" (UniqueName: \"kubernetes.io/projected/6e5a33b9-dc45-4935-981f-8089cc2cd7d3-kube-api-access-nzsnh\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:26.193086 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:26.193046    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-945b9b\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:26.195756 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.195724    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" path="/var/lib/kubelet/pods/6e5a33b9-dc45-4935-981f-8089cc2cd7d3/volumes"
Sep 14 05:08:26.196256 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.196234    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" path="/var/lib/kubelet/pods/cb0fb9c3-ffee-4fe9-aa93-2b0f51902643/volumes"
Sep 14 05:08:26.438141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.438057    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-2aa8e5"]
Sep 14 05:08:26.444461 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.444433    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-2aa8e5"]
Sep 14 05:08:26.574726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.574679    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7533-85ca11"]
Sep 14 05:08:26.575318 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575301    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" containerName="maas-api"
Sep 14 05:08:26.575380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575321    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" containerName="maas-api"
Sep 14 05:08:26.575380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575330    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" containerName="test-curl-7533-2aa8e5"
Sep 14 05:08:26.575380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575338    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" containerName="test-curl-7533-2aa8e5"
Sep 14 05:08:26.575477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575408    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb0fb9c3-ffee-4fe9-aa93-2b0f51902643" containerName="maas-api"
Sep 14 05:08:26.575477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.575417    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" containerName="test-curl-7533-2aa8e5"
Sep 14 05:08:26.580418 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.580391    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:26.582010 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.581983    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-85ca11"]
Sep 14 05:08:26.655806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.655762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:26.680104 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.680060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-952gf\" (UniqueName: \"kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf\") pod \"test-curl-7533-85ca11\" (UID: \"c18093c2-99ed-47ee-a484-8c0d6fd28058\") " pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:26.781020 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.780984    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-952gf\" (UniqueName: \"kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf\") pod \"test-curl-7533-85ca11\" (UID: \"c18093c2-99ed-47ee-a484-8c0d6fd28058\") " pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:26.789204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.789166    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-952gf\" (UniqueName: \"kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf\") pod \"test-curl-7533-85ca11\" (UID: \"c18093c2-99ed-47ee-a484-8c0d6fd28058\") " pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:26.804346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.804299    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e5a33b9-dc45-4935-981f-8089cc2cd7d3" containerID="f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd" exitCode=137
Sep 14 05:08:26.804346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.804341    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-576b01"
Sep 14 05:08:26.804872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.804398    2571 scope.go:117] "RemoveContainer" containerID="f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd"
Sep 14 05:08:26.804872 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:26.804677    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7533-2aa8e5" container="test-curl-7533-2aa8e5"
Sep 14 05:08:26.813508 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.813484    2571 scope.go:117] "RemoveContainer" containerID="f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd"
Sep 14 05:08:26.813867 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:26.813847    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd\": container with ID starting with f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd not found: ID does not exist" containerID="f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd"
Sep 14 05:08:26.813920 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.813879    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd"} err="failed to get container status \"f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd\": rpc error: code = NotFound desc = could not find container \"f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd\": container with ID starting with f17f417e317acc28484fdfc428e6bb0b239cb2259231c8f4880544ee2ffce9fd not found: ID does not exist"
Sep 14 05:08:26.892188 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:26.892137    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:27.167816 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.167789    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7533-85ca11"]
Sep 14 05:08:27.170771 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:27.170715    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc18093c2_99ed_47ee_a484_8c0d6fd28058.slice/crio-90d514b6daf392723e950907d29a39dd52eeff402a9027de91a0af7c20edb394 WatchSource:0}: Error finding container 90d514b6daf392723e950907d29a39dd52eeff402a9027de91a0af7c20edb394: Status 404 returned error can't find the container with id 90d514b6daf392723e950907d29a39dd52eeff402a9027de91a0af7c20edb394
Sep 14 05:08:27.352508 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.352446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:27.812519 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.812482    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-85ca11" event={"ID":"c18093c2-99ed-47ee-a484-8c0d6fd28058","Type":"ContainerStarted","Data":"2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1"}
Sep 14 05:08:27.812917 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.812537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7533-85ca11" event={"ID":"c18093c2-99ed-47ee-a484-8c0d6fd28058","Type":"ContainerStarted","Data":"90d514b6daf392723e950907d29a39dd52eeff402a9027de91a0af7c20edb394"}
Sep 14 05:08:27.813702 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.813667    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7533-2aa8e5" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" containerName="test-curl-7533-2aa8e5" containerID="cri-o://364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc" gracePeriod=2
Sep 14 05:08:27.827184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:27.827135    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7533-85ca11" podStartSLOduration=1.8271181250000001 podStartE2EDuration="1.827118125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:27.825893101 +0000 UTC m=+1302.187845547" watchObservedRunningTime="2026-09-14 05:08:27.827118125 +0000 UTC m=+1302.189070567"
Sep 14 05:08:27.828118 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:27.828087    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-2aa8e5\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:28.003237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.003208    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:28.005289 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:28.005261    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-2aa8e5\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:28.040865 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.040829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:28.094412 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.094318    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlpxf\" (UniqueName: \"kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf\") pod \"fc1c095b-1f97-444b-8623-2b057d4f38e5\" (UID: \"fc1c095b-1f97-444b-8623-2b057d4f38e5\") "
Sep 14 05:08:28.096610 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.096580    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf" (OuterVolumeSpecName: "kube-api-access-jlpxf") pod "fc1c095b-1f97-444b-8623-2b057d4f38e5" (UID: "fc1c095b-1f97-444b-8623-2b057d4f38e5"). InnerVolumeSpecName "kube-api-access-jlpxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:08:28.194880 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.194843    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc1c095b-1f97-444b-8623-2b057d4f38e5" path="/var/lib/kubelet/pods/fc1c095b-1f97-444b-8623-2b057d4f38e5/volumes"
Sep 14 05:08:28.195784 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.195757    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlpxf\" (UniqueName: \"kubernetes.io/projected/fc1c095b-1f97-444b-8623-2b057d4f38e5-kube-api-access-jlpxf\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:28.438204 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.438080    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7533-85ca11"]
Sep 14 05:08:28.441834 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.441803    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7533-85ca11"]
Sep 14 05:08:28.821032 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.820993    2571 generic.go:356] "Generic (PLEG): container finished" podID="fc1c095b-1f97-444b-8623-2b057d4f38e5" containerID="79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f" exitCode=137
Sep 14 05:08:28.821632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.821038    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-945b9b"
Sep 14 05:08:28.821632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.821112    2571 scope.go:117] "RemoveContainer" containerID="79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f"
Sep 14 05:08:28.821632 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:28.821319    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7533-85ca11" container="test-curl-7533-85ca11"
Sep 14 05:08:28.830425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.830402    2571 scope.go:117] "RemoveContainer" containerID="79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f"
Sep 14 05:08:28.830773 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:28.830753    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f\": container with ID starting with 79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f not found: ID does not exist" containerID="79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f"
Sep 14 05:08:28.830822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.830783    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f"} err="failed to get container status \"79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f\": rpc error: code = NotFound desc = could not find container \"79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f\": container with ID starting with 79ad69578362ea2ba5d74d04176406638eceb84a90407c4fd04e01d4bb02d45f not found: ID does not exist"
Sep 14 05:08:28.959791 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:28.959756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:29.067446 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:29.067408    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:29.366367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:29.366336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:29.682604 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:29.682516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:29.828414 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:29.828370    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7533-85ca11" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" containerName="test-curl-7533-85ca11" containerID="cri-o://2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1" gracePeriod=2
Sep 14 05:08:29.831368 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:29.831327    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-85ca11\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:29.984597 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:29.984564    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:29.986733 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:29.986699    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7533-85ca11\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:30.115045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.115001    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p8wq\" (UniqueName: \"kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq\") pod \"8df1316a-9f3a-46fd-b17d-fdf847b0ab64\" (UID: \"8df1316a-9f3a-46fd-b17d-fdf847b0ab64\") "
Sep 14 05:08:30.117376 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.117346    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq" (OuterVolumeSpecName: "kube-api-access-2p8wq") pod "8df1316a-9f3a-46fd-b17d-fdf847b0ab64" (UID: "8df1316a-9f3a-46fd-b17d-fdf847b0ab64"). InnerVolumeSpecName "kube-api-access-2p8wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:08:30.201734 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.201689    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" path="/var/lib/kubelet/pods/8df1316a-9f3a-46fd-b17d-fdf847b0ab64/volumes"
Sep 14 05:08:30.216494 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.216457    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p8wq\" (UniqueName: \"kubernetes.io/projected/8df1316a-9f3a-46fd-b17d-fdf847b0ab64-kube-api-access-2p8wq\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:30.836151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.836110    2571 generic.go:356] "Generic (PLEG): container finished" podID="8df1316a-9f3a-46fd-b17d-fdf847b0ab64" containerID="364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc" exitCode=137
Sep 14 05:08:30.836612 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.836168    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-2aa8e5"
Sep 14 05:08:30.836612 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.836172    2571 scope.go:117] "RemoveContainer" containerID="364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc"
Sep 14 05:08:30.845367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.845340    2571 scope.go:117] "RemoveContainer" containerID="364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc"
Sep 14 05:08:30.845732 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:30.845707    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc\": container with ID starting with 364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc not found: ID does not exist" containerID="364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc"
Sep 14 05:08:30.845788 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:30.845745    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc"} err="failed to get container status \"364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc\": rpc error: code = NotFound desc = could not find container \"364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc\": container with ID starting with 364f203e2ad3c92c815dd0a7e7f62c92c786afd2bc74620421ff7a0c766787fc not found: ID does not exist"
Sep 14 05:08:31.163603 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:31.163507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:31.231247 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:31.231219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:31.461034 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:31.461002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:31.953512 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:31.953472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:32.021796 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.021764    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:32.053741 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.053705    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:32.134521 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.134425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-952gf\" (UniqueName: \"kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf\") pod \"c18093c2-99ed-47ee-a484-8c0d6fd28058\" (UID: \"c18093c2-99ed-47ee-a484-8c0d6fd28058\") "
Sep 14 05:08:32.136758 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.136733    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf" (OuterVolumeSpecName: "kube-api-access-952gf") pod "c18093c2-99ed-47ee-a484-8c0d6fd28058" (UID: "c18093c2-99ed-47ee-a484-8c0d6fd28058"). InnerVolumeSpecName "kube-api-access-952gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:08:32.195242 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.195207    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" path="/var/lib/kubelet/pods/c18093c2-99ed-47ee-a484-8c0d6fd28058/volumes"
Sep 14 05:08:32.236184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.236148    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-952gf\" (UniqueName: \"kubernetes.io/projected/c18093c2-99ed-47ee-a484-8c0d6fd28058-kube-api-access-952gf\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:08:32.284173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.284135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:32.611627 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.611582    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:32.711405 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.711369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:32.849830 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.849792    2571 generic.go:356] "Generic (PLEG): container finished" podID="c18093c2-99ed-47ee-a484-8c0d6fd28058" containerID="2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1" exitCode=137
Sep 14 05:08:32.850039 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.849846    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7533-85ca11"
Sep 14 05:08:32.850039 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.849899    2571 scope.go:117] "RemoveContainer" containerID="2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1"
Sep 14 05:08:32.861985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.861961    2571 scope.go:117] "RemoveContainer" containerID="2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1"
Sep 14 05:08:32.862392 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:32.862368    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1\": container with ID starting with 2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1 not found: ID does not exist" containerID="2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1"
Sep 14 05:08:32.862479 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.862402    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1"} err="failed to get container status \"2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1\": rpc error: code = NotFound desc = could not find container \"2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1\": container with ID starting with 2f0527dff91df5ba0980018dffc06ab8384dd020544feb769c993e8f2cf815e1 not found: ID does not exist"
Sep 14 05:08:32.904583 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:32.904551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:33.705151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:33.705119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:33.945632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:33.945593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:34.044464 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:34.044376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:34.163746 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:34.163700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:34.549987 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:34.549956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:35.342381 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:35.342327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:36.048673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.048641    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:36.830093 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.830059    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:08:36.830573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.830554    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" containerName="test-curl-7533-85ca11"
Sep 14 05:08:36.830634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.830575    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" containerName="test-curl-7533-85ca11"
Sep 14 05:08:36.830670 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.830641    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c18093c2-99ed-47ee-a484-8c0d6fd28058" containerName="test-curl-7533-85ca11"
Sep 14 05:08:36.835284 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.835263    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:36.837729 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.837703    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c7c6a48c-serving-cert\""
Sep 14 05:08:36.842167 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.842136    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:08:36.857365 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.857327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:36.910574 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.910539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:36.987879 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.987837    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:36.988105 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.987901    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:36.988105 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.988046    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:36.988105 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:36.988095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnlnb\" (UniqueName: \"kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.089160 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.089074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.089160 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.089145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.089384 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.089180    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.089384 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.089201    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnlnb\" (UniqueName: \"kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.089384 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:37.089324    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-serving-cert: secret "maas-api-e2e-mt-c7c6a48c-serving-cert" not found
Sep 14 05:08:37.089503 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:08:37.089398    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls podName:5f685f36-4049-4125-8995-ebb959a98c90 nodeName:}" failed. No retries permitted until 2026-09-14 05:08:37.58937634 +0000 UTC m=+1311.951328761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls") pod "maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" (UID: "5f685f36-4049-4125-8995-ebb959a98c90") : secret "maas-api-e2e-mt-c7c6a48c-serving-cert" not found
Sep 14 05:08:37.089862 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.089839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.091781 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.091758    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.099678 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.099647    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnlnb\" (UniqueName: \"kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.254426 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.254388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:37.357631 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.357529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:37.456447 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.456407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:37.593910 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.593865    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.596710 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.596680    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") pod \"maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:37.750620 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:37.750584    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:38.027877 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:38.027801    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:08:38.030391 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:08:38.030360    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f685f36_4049_4125_8995_ebb959a98c90.slice/crio-7e87d6918a42c3d94de4e7991a3b5c79769562a9a91059aa99be8b6f9ad0058f WatchSource:0}: Error finding container 7e87d6918a42c3d94de4e7991a3b5c79769562a9a91059aa99be8b6f9ad0058f: Status 404 returned error can't find the container with id 7e87d6918a42c3d94de4e7991a3b5c79769562a9a91059aa99be8b6f9ad0058f
Sep 14 05:08:38.887021 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:38.886889    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" event={"ID":"5f685f36-4049-4125-8995-ebb959a98c90","Type":"ContainerStarted","Data":"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"}
Sep 14 05:08:38.887021 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:38.886981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" event={"ID":"5f685f36-4049-4125-8995-ebb959a98c90","Type":"ContainerStarted","Data":"7e87d6918a42c3d94de4e7991a3b5c79769562a9a91059aa99be8b6f9ad0058f"}
Sep 14 05:08:38.887021 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:38.887006    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:38.903350 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:38.903159    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" podStartSLOduration=2.334509398 podStartE2EDuration="2.903136447s" totalImagesPullingTime="568.627049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:08:38.902317558 +0000 UTC m=+1313.264270002" watchObservedRunningTime="2026-09-14 05:08:38.903136447 +0000 UTC m=+1313.265088895"
Sep 14 05:08:44.898001 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:44.897971    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:08:56.833593 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:56.833549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:58.952403 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:58.952363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:08:59.636172 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:08:59.636132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:00.492012 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:00.491969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:01.271679 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:01.271643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:01.893697 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:01.893661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:02.636144 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:02.636105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:02.816684 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:02.816646    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:03.459348 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:03.459307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:08.762257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:08.762169    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:10.828666 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:10.828630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:11.051001 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:11.050967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:14.378394 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:14.378357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:44.220595 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:44.220553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:45.469219 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:45.469177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:46.243627 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:46.243566    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:46.882621 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:46.882581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:47.135306 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:47.135213    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:47.235683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:47.235648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:47.335867 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:47.335834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:47.776845 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:47.776806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:47.833183 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:47.833144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:48.467208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:48.467176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:48.623438 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:48.623405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:49.449470 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:49.449436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:49.834186 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:49.834156    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:50.134011 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:50.133888    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:50.786962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:50.786903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:52.335918 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:52.335868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:55.755633 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:55.755600    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:09:55.756112 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:55.755839    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" podUID="fd4a2b79-fa56-479c-a99b-850045848771" containerName="maas-api" containerID="cri-o://d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b" gracePeriod=30
Sep 14 05:09:55.782621 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:55.782570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:55.882717 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:55.882672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:55.971381 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:55.971294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:56.018671 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.018596    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:09:56.033469 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.033434    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:56.191212 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191183    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnwt8\" (UniqueName: \"kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8\") pod \"fd4a2b79-fa56-479c-a99b-850045848771\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") "
Sep 14 05:09:56.191391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191220    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca\") pod \"fd4a2b79-fa56-479c-a99b-850045848771\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") "
Sep 14 05:09:56.191391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191349    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls\") pod \"fd4a2b79-fa56-479c-a99b-850045848771\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") "
Sep 14 05:09:56.191391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191386    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs\") pod \"fd4a2b79-fa56-479c-a99b-850045848771\" (UID: \"fd4a2b79-fa56-479c-a99b-850045848771\") "
Sep 14 05:09:56.191648 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191616    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd4a2b79-fa56-479c-a99b-850045848771" (UID: "fd4a2b79-fa56-479c-a99b-850045848771"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:09:56.191775 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.191758    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4a2b79-fa56-479c-a99b-850045848771-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:09:56.193699 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.193668    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd4a2b79-fa56-479c-a99b-850045848771" (UID: "fd4a2b79-fa56-479c-a99b-850045848771"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:09:56.193815 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.193757    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8" (OuterVolumeSpecName: "kube-api-access-gnwt8") pod "fd4a2b79-fa56-479c-a99b-850045848771" (UID: "fd4a2b79-fa56-479c-a99b-850045848771"). InnerVolumeSpecName "kube-api-access-gnwt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:09:56.194084 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.194058    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fd4a2b79-fa56-479c-a99b-850045848771" (UID: "fd4a2b79-fa56-479c-a99b-850045848771"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:09:56.292602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.292514    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnwt8\" (UniqueName: \"kubernetes.io/projected/fd4a2b79-fa56-479c-a99b-850045848771-kube-api-access-gnwt8\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:09:56.292602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.292556    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:09:56.292602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.292572    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd4a2b79-fa56-479c-a99b-850045848771-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:09:56.334312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.334281    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd4a2b79-fa56-479c-a99b-850045848771" containerID="d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b" exitCode=0
Sep 14 05:09:56.334485 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.334329    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"
Sep 14 05:09:56.334485 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.334367    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" event={"ID":"fd4a2b79-fa56-479c-a99b-850045848771","Type":"ContainerDied","Data":"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"}
Sep 14 05:09:56.334485 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.334428    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg" event={"ID":"fd4a2b79-fa56-479c-a99b-850045848771","Type":"ContainerDied","Data":"0d5f269eaa153a3e961ecbfa4d508d5737175b8257c5df8000ee61f176e767a7"}
Sep 14 05:09:56.334485 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.334453    2571 scope.go:117] "RemoveContainer" containerID="d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"
Sep 14 05:09:56.343776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.343757    2571 scope.go:117] "RemoveContainer" containerID="d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"
Sep 14 05:09:56.344083 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:09:56.344063    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b\": container with ID starting with d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b not found: ID does not exist" containerID="d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"
Sep 14 05:09:56.344166 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.344094    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b"} err="failed to get container status \"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b\": rpc error: code = NotFound desc = could not find container \"d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b\": container with ID starting with d8df711bdad1a5a08894f151d3d0fc7bc894790b39e01e474b02c0b38bba6a6b not found: ID does not exist"
Sep 14 05:09:56.349795 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.349771    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:09:56.353419 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.353393    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-667664-7f5ccbc74c-kktsg"]
Sep 14 05:09:56.374269 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.374231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:56.427973 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:56.427911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:57.067705 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:57.067633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:57.863248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:57.863207    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:58.195441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:58.195357    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4a2b79-fa56-479c-a99b-850045848771" path="/var/lib/kubelet/pods/fd4a2b79-fa56-479c-a99b-850045848771/volumes"
Sep 14 05:09:58.637823 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:58.637786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:09:59.279591 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:09:59.279553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:00.058302 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:00.058270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:02.375875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:02.375822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:02.476345 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:02.476308    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:02.681330 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:02.681239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:03.361844 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:03.361814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:04.163663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:04.163611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:04.846872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:04.846834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:05.523667 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:05.523618    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:06.251983 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:06.251950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:06.882410 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:06.882376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:07.633843 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:07.633813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:09.340391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:09.340355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:11.341180 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:11.341128    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:10:11.361495 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:11.361464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:13.578921 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.578880    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:10:13.579488 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.579198    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" podUID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" containerName="maas-api" containerID="cri-o://adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157" gracePeriod=30
Sep 14 05:10:13.601579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.601532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:13.701153 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.701123    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:13.792092 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.792054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:13.839919 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.839852    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:10:13.843505 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.843477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:13.969861 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.969823    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgf5s\" (UniqueName: \"kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s\") pod \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") "
Sep 14 05:10:13.969861 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.969877    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") pod \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") "
Sep 14 05:10:13.970148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.970024    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs\") pod \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") "
Sep 14 05:10:13.970148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.970068    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca\") pod \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\" (UID: \"e550c7d0-4ebd-45a9-a42b-5c363afdc266\") "
Sep 14 05:10:13.970726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.970696    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e550c7d0-4ebd-45a9-a42b-5c363afdc266" (UID: "e550c7d0-4ebd-45a9-a42b-5c363afdc266"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:10:13.973210 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.973163    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e550c7d0-4ebd-45a9-a42b-5c363afdc266" (UID: "e550c7d0-4ebd-45a9-a42b-5c363afdc266"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:10:13.973434 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.973398    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e550c7d0-4ebd-45a9-a42b-5c363afdc266" (UID: "e550c7d0-4ebd-45a9-a42b-5c363afdc266"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:10:13.973622 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:13.973595    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s" (OuterVolumeSpecName: "kube-api-access-kgf5s") pod "e550c7d0-4ebd-45a9-a42b-5c363afdc266" (UID: "e550c7d0-4ebd-45a9-a42b-5c363afdc266"). InnerVolumeSpecName "kube-api-access-kgf5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:10:14.071797 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.071753    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgf5s\" (UniqueName: \"kubernetes.io/projected/e550c7d0-4ebd-45a9-a42b-5c363afdc266-kube-api-access-kgf5s\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:14.072073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.072044    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:14.072073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.072070    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e550c7d0-4ebd-45a9-a42b-5c363afdc266-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:14.072225 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.072084    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e550c7d0-4ebd-45a9-a42b-5c363afdc266-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:14.224809 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.224772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:14.325278 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.325242    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:14.445888 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.445848    2571 generic.go:356] "Generic (PLEG): container finished" podID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" containerID="adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157" exitCode=0
Sep 14 05:10:14.445888 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.445895    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"
Sep 14 05:10:14.446151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.445895    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" event={"ID":"e550c7d0-4ebd-45a9-a42b-5c363afdc266","Type":"ContainerDied","Data":"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"}
Sep 14 05:10:14.446151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.445985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl" event={"ID":"e550c7d0-4ebd-45a9-a42b-5c363afdc266","Type":"ContainerDied","Data":"b1b52a888e187809155863751269778f4efb43144909749cf160eaa0659506c0"}
Sep 14 05:10:14.446151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.446017    2571 scope.go:117] "RemoveContainer" containerID="adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"
Sep 14 05:10:14.455903 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.455877    2571 scope.go:117] "RemoveContainer" containerID="adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"
Sep 14 05:10:14.456265 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:14.456238    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157\": container with ID starting with adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157 not found: ID does not exist" containerID="adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"
Sep 14 05:10:14.456346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.456279    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157"} err="failed to get container status \"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157\": rpc error: code = NotFound desc = could not find container \"adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157\": container with ID starting with adfd255b3043b5a5b9fa121e6d26851e7f3ad09a52de6cc33de4c424505ed157 not found: ID does not exist"
Sep 14 05:10:14.463398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.463354    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:10:14.467731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.467703    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e7fdd6-79b9887d6c-rjgtl"]
Sep 14 05:10:14.998916 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:14.998878    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:15.750132 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:15.750096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:16.194726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:16.194635    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" path="/var/lib/kubelet/pods/e550c7d0-4ebd-45a9-a42b-5c363afdc266/volumes"
Sep 14 05:10:16.295271 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:16.295238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:16.458053 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:16.458018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:17.100066 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:17.100035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:17.730355 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:17.730304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:20.223391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:20.223348    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:20.278659 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:20.278624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:20.522235 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:20.522142    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:21.184502 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:21.184464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:21.625794 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:21.625741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:21.746397 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:21.746361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:23.385192 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:23.385158    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:23.554566 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:23.554530    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:24.378286 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:24.378204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:24.477249 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:24.477220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:24.576673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:24.576635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:25.271519 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:25.271481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:25.951941 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:25.951906    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:26.362875 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.362783    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" podUID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" containerName="main" containerID="cri-o://d25748f0e2a02a7fa09367b5962585bb3c6e108fb2c3365b6c96457d2f1881f0" gracePeriod=45
Sep 14 05:10:26.516814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.516781    2571 generic.go:356] "Generic (PLEG): container finished" podID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" containerID="d25748f0e2a02a7fa09367b5962585bb3c6e108fb2c3365b6c96457d2f1881f0" exitCode=0
Sep 14 05:10:26.517039 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.516822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" event={"ID":"00767c2a-9dfa-4141-8c73-691fdc3aedd6","Type":"ContainerDied","Data":"d25748f0e2a02a7fa09367b5962585bb3c6e108fb2c3365b6c96457d2f1881f0"}
Sep 14 05:10:26.638687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.638657    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:10:26.658748 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.658712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:26.797257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797222    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797268    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797511 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797321    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v6dz\" (UniqueName: \"kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797511 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797511 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797410    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797511 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797442    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir\") pod \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\" (UID: \"00767c2a-9dfa-4141-8c73-691fdc3aedd6\") "
Sep 14 05:10:26.797719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797629    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home" (OuterVolumeSpecName: "home") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:10:26.797781 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797740    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache" (OuterVolumeSpecName: "model-cache") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:10:26.797838 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797777    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-home\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:26.797838 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.797790    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:10:26.799715 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.799685    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz" (OuterVolumeSpecName: "kube-api-access-7v6dz") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "kube-api-access-7v6dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:10:26.800131 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.800105    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:10:26.800131 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.800113    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm" (OuterVolumeSpecName: "dshm") pod "00767c2a-9dfa-4141-8c73-691fdc3aedd6" (UID: "00767c2a-9dfa-4141-8c73-691fdc3aedd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:10:26.898814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.898719    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-dshm\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:26.898814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.898755    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tls-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:26.898814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.898766    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v6dz\" (UniqueName: \"kubernetes.io/projected/00767c2a-9dfa-4141-8c73-691fdc3aedd6-kube-api-access-7v6dz\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:26.898814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.898774    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-model-cache\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:26.898814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:26.898783    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00767c2a-9dfa-4141-8c73-691fdc3aedd6-tmp-dir\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:27.345002 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.344970    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345659    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345684    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345712    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4a2b79-fa56-479c-a99b-850045848771" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345722    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4a2b79-fa56-479c-a99b-850045848771" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345742    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" containerName="main"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345752    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" containerName="main"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345872    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e550c7d0-4ebd-45a9-a42b-5c363afdc266" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345887    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4a2b79-fa56-479c-a99b-850045848771" containerName="maas-api"
Sep 14 05:10:27.345962 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.345897    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" containerName="main"
Sep 14 05:10:27.349777 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.349751    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.352052 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.352032    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-2ff4a6-serving-cert\""
Sep 14 05:10:27.362471 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.361113    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:10:27.377487 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.377448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:27.430241 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.430209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:27.504731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.504691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.504731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.504733    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.504982 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.504792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.504982 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.504846    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mssk\" (UniqueName: \"kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.523852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.523824    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"
Sep 14 05:10:27.524079 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.523850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz" event={"ID":"00767c2a-9dfa-4141-8c73-691fdc3aedd6","Type":"ContainerDied","Data":"e898f690a475657082708ea8e152e687ead29f9d439eaca776eacae9e462f712"}
Sep 14 05:10:27.524079 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.523900    2571 scope.go:117] "RemoveContainer" containerID="d25748f0e2a02a7fa09367b5962585bb3c6e108fb2c3365b6c96457d2f1881f0"
Sep 14 05:10:27.547124 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.547078    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:10:27.549240 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.549216    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e08402/e2e-conflict-a-c6882fdf-kserve-58c6699b7b-dsttz"]
Sep 14 05:10:27.605765 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.605663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.605765 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.605703    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.605765 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.605735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.606050 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.605777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mssk\" (UniqueName: \"kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.606050 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:27.605854    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-serving-cert: secret "maas-api-e2e-shared-a-w3-2ff4a6-serving-cert" not found
Sep 14 05:10:27.606050 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:27.605916    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls podName:4b882d5f-a5c3-4781-b492-b7a117da1afb nodeName:}" failed. No retries permitted until 2026-09-14 05:10:28.105899057 +0000 UTC m=+1422.467851478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls") pod "maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" (UID: "4b882d5f-a5c3-4781-b492-b7a117da1afb") : secret "maas-api-e2e-shared-a-w3-2ff4a6-serving-cert" not found
Sep 14 05:10:27.606535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.606515    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.608381 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.608359    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.614331 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.614302    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mssk\" (UniqueName: \"kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:27.939722 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:27.939639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:28.033418 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.033378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:28.111045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.110996    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:28.111486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.111462    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:28.113594 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.113573    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:28.195228 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.195146    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00767c2a-9dfa-4141-8c73-691fdc3aedd6" path="/var/lib/kubelet/pods/00767c2a-9dfa-4141-8c73-691fdc3aedd6/volumes"
Sep 14 05:10:28.265316 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.265285    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:28.547732 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:28.547705    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:10:28.549376 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:10:28.549351    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b882d5f_a5c3_4781_b492_b7a117da1afb.slice/crio-b70fb6de6432b733dee0e8f7ee38c5578a79754d26d454e03772d0ee67c7d1c3 WatchSource:0}: Error finding container b70fb6de6432b733dee0e8f7ee38c5578a79754d26d454e03772d0ee67c7d1c3: Status 404 returned error can't find the container with id b70fb6de6432b733dee0e8f7ee38c5578a79754d26d454e03772d0ee67c7d1c3
Sep 14 05:10:29.525337 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.525274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:29.538560 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.538522    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" event={"ID":"4b882d5f-a5c3-4781-b492-b7a117da1afb","Type":"ContainerStarted","Data":"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"}
Sep 14 05:10:29.538776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.538611    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" event={"ID":"4b882d5f-a5c3-4781-b492-b7a117da1afb","Type":"ContainerStarted","Data":"b70fb6de6432b733dee0e8f7ee38c5578a79754d26d454e03772d0ee67c7d1c3"}
Sep 14 05:10:29.538776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.538647    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:29.557410 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.557352    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" podStartSLOduration=2.258914732 podStartE2EDuration="2.557333069s" totalImagesPullingTime="298.418337ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:10:29.556348538 +0000 UTC m=+1423.918300981" watchObservedRunningTime="2026-09-14 05:10:29.557333069 +0000 UTC m=+1423.919285511"
Sep 14 05:10:29.992025 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:29.991979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:30.662783 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:30.662746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:31.460624 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:31.460591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:32.124079 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:32.124045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:32.763965 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:32.763912    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:33.478787 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:33.478756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:35.557305 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:35.557270    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:10:39.670695 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:39.670608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.206854 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.206793    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:10:45.207704 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.207664    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" podUID="5f685f36-4049-4125-8995-ebb959a98c90" containerName="maas-api" containerID="cri-o://7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515" gracePeriod=30
Sep 14 05:10:45.230913 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.230879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.299560 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.299514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.404233 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.404187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.466237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.466199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.479262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.479227    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:10:45.582159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.582122    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") pod \"5f685f36-4049-4125-8995-ebb959a98c90\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") "
Sep 14 05:10:45.582159 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.582168    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnlnb\" (UniqueName: \"kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb\") pod \"5f685f36-4049-4125-8995-ebb959a98c90\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") "
Sep 14 05:10:45.582428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.582206    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs\") pod \"5f685f36-4049-4125-8995-ebb959a98c90\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") "
Sep 14 05:10:45.582428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.582236    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca\") pod \"5f685f36-4049-4125-8995-ebb959a98c90\" (UID: \"5f685f36-4049-4125-8995-ebb959a98c90\") "
Sep 14 05:10:45.582694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.582658    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f685f36-4049-4125-8995-ebb959a98c90" (UID: "5f685f36-4049-4125-8995-ebb959a98c90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:10:45.584621 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.584593    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5f685f36-4049-4125-8995-ebb959a98c90" (UID: "5f685f36-4049-4125-8995-ebb959a98c90"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:10:45.584762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.584618    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb" (OuterVolumeSpecName: "kube-api-access-lnlnb") pod "5f685f36-4049-4125-8995-ebb959a98c90" (UID: "5f685f36-4049-4125-8995-ebb959a98c90"). InnerVolumeSpecName "kube-api-access-lnlnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:10:45.584824 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.584777    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f685f36-4049-4125-8995-ebb959a98c90" (UID: "5f685f36-4049-4125-8995-ebb959a98c90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:10:45.628385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.628351    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f685f36-4049-4125-8995-ebb959a98c90" containerID="7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515" exitCode=0
Sep 14 05:10:45.628385 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.628390    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"
Sep 14 05:10:45.628656 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.628466    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" event={"ID":"5f685f36-4049-4125-8995-ebb959a98c90","Type":"ContainerDied","Data":"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"}
Sep 14 05:10:45.628656 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.628532    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg" event={"ID":"5f685f36-4049-4125-8995-ebb959a98c90","Type":"ContainerDied","Data":"7e87d6918a42c3d94de4e7991a3b5c79769562a9a91059aa99be8b6f9ad0058f"}
Sep 14 05:10:45.628656 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.628564    2571 scope.go:117] "RemoveContainer" containerID="7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"
Sep 14 05:10:45.639287 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.639262    2571 scope.go:117] "RemoveContainer" containerID="7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"
Sep 14 05:10:45.639633 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:45.639611    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515\": container with ID starting with 7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515 not found: ID does not exist" containerID="7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"
Sep 14 05:10:45.639715 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.639648    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515"} err="failed to get container status \"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515\": rpc error: code = NotFound desc = could not find container \"7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515\": container with ID starting with 7f81f0dfc2496a6cd24ab83c0fdb75602da4a97f0cf02b5aaf9ff08b5bf98515 not found: ID does not exist"
Sep 14 05:10:45.652253 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.652207    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:10:45.654079 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.654054    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c7c6a48c-84dfc85555-67xlg"]
Sep 14 05:10:45.683731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.683691    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:45.683731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.683724    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnlnb\" (UniqueName: \"kubernetes.io/projected/5f685f36-4049-4125-8995-ebb959a98c90-kube-api-access-lnlnb\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:45.683731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.683735    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f685f36-4049-4125-8995-ebb959a98c90-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:45.684002 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.683746    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f685f36-4049-4125-8995-ebb959a98c90-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:10:45.861600 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.861512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:45.939916 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:45.939878    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:46.195734 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:46.195657    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f685f36-4049-4125-8995-ebb959a98c90" path="/var/lib/kubelet/pods/5f685f36-4049-4125-8995-ebb959a98c90/volumes"
Sep 14 05:10:46.685010 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:46.684969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:47.357684 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:47.357643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:48.260417 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:48.260381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:48.986097 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:48.986063    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:49.147347 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:49.147315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:49.457286 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:49.457256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:49.835223 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:49.835133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:49.943377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:49.943341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:50.143675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:50.143590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:50.554407 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:50.554367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:50.773122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:50.773084    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:51.548410 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:51.548369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:52.265912 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:52.265879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:53.064648 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:53.064607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:53.487977 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:53.487919    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:53.582976 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:53.582925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:53.857748 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:53.857649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:54.067873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:54.067823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:54.570311 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:54.570274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:55.328040 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:55.328005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:56.164763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.164730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:56.822178 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.822142    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:10:56.822836 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.822811    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f685f36-4049-4125-8995-ebb959a98c90" containerName="maas-api"
Sep 14 05:10:56.823057 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.823030    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f685f36-4049-4125-8995-ebb959a98c90" containerName="maas-api"
Sep 14 05:10:56.823365 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.823343    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f685f36-4049-4125-8995-ebb959a98c90" containerName="maas-api"
Sep 14 05:10:56.827905 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.827880    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.830852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.830829    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-27be50-serving-cert\""
Sep 14 05:10:56.838882 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.838855    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:10:56.857676 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.857634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:56.892835 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.892793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bprzj\" (UniqueName: \"kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.893064 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.892844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.893064 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.892925    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.893064 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.893005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.958873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.958840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:56.994329 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.994276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.994531 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.994346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.994531 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.994381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.994531 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.994498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bprzj\" (UniqueName: \"kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.994702 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:56.994581    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-serving-cert: secret "maas-api-e2e-shared-b-w3-27be50-serving-cert" not found
Sep 14 05:10:56.994702 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:10:56.994647    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls podName:e0de8bc9-8a60-41c9-8a3a-f1e028a23e47 nodeName:}" failed. No retries permitted until 2026-09-14 05:10:57.494632114 +0000 UTC m=+1451.856584535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls") pod "maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" (UID: "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47") : secret "maas-api-e2e-shared-b-w3-27be50-serving-cert" not found
Sep 14 05:10:56.995238 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.995217    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:56.996945 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:56.996913    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:57.002995 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.002962    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bprzj\" (UniqueName: \"kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:57.203496 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.203464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:57.303816 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.303777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:57.406260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.406226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:10:57.501241 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.501128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:57.503846 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.503817    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:57.741296 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:57.741260    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:58.020723 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:58.020695    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:10:58.023312 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:10:58.023274    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0de8bc9_8a60_41c9_8a3a_f1e028a23e47.slice/crio-6eeeb7389b16e896de4ea1e052dbe66cfe874017115afa651ab4c8f0b82f3923 WatchSource:0}: Error finding container 6eeeb7389b16e896de4ea1e052dbe66cfe874017115afa651ab4c8f0b82f3923: Status 404 returned error can't find the container with id 6eeeb7389b16e896de4ea1e052dbe66cfe874017115afa651ab4c8f0b82f3923
Sep 14 05:10:58.711772 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:58.711664    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" event={"ID":"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47","Type":"ContainerStarted","Data":"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"}
Sep 14 05:10:58.712229 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:58.711846    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:10:58.712229 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:58.711873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" event={"ID":"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47","Type":"ContainerStarted","Data":"6eeeb7389b16e896de4ea1e052dbe66cfe874017115afa651ab4c8f0b82f3923"}
Sep 14 05:10:58.729362 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:10:58.729306    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" podStartSLOduration=2.2937292080000002 podStartE2EDuration="2.729289858s" totalImagesPullingTime="435.56065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:10:58.727713374 +0000 UTC m=+1453.089665817" watchObservedRunningTime="2026-09-14 05:10:58.729289858 +0000 UTC m=+1453.091242300"
Sep 14 05:11:04.724753 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:04.724721    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:11:19.743620 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:19.743550    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:20.481756 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:20.481706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:21.264375 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:21.264335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:21.951433 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:21.951372    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:22.651423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:22.651383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:23.444006 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:23.443967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:23.731325 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:23.731288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:26.542616 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:26.542574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:30.612366 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:30.612331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:31.303060 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:31.303001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:37.150856 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:37.150810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:11:46.228185 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:46.228154    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:11:46.230538 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:46.230514    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:11:46.233851 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:46.233832    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:11:46.235734 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:46.235716    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:11:58.054547 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:11:58.054482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:09.762262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:09.762176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:09.862537 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:09.862498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:10.035586 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:10.035500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:11.385219 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:11.385181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:11.480633 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:11.480596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:11.580554 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:11.580493    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:12.273806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:12.273733    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:12.375047 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:12.375000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:12.474074 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:12.474041    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:13.826952 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:13.826883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:15.723326 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.723293    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:12:15.727631 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.727603    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.731247 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.731213    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-27be50\"/\"kube-root-ca.crt\""
Sep 14 05:12:15.731956 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.731250    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-27be50\"/\"auth-test-model-27be50-kserve-self-signed-certs\""
Sep 14 05:12:15.731956 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.731687    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-27be50\"/\"openshift-service-ca.crt\""
Sep 14 05:12:15.731956 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.731841    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-27be50\"/\"default-dockercfg-m8bz9\""
Sep 14 05:12:15.734516 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.734492    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:12:15.746763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.746711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:15.843148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.843303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843191    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.843303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843211    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwpp9\" (UniqueName: \"kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.843303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.843303 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843259    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.843463 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.843306    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.943883 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.943823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.943920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.943987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwpp9\" (UniqueName: \"kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944136 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944401 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944309    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944401 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944334    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.944516 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.944477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.946389 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.946368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.946551 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.946532    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:15.951980 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:15.951954    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwpp9\" (UniqueName: \"kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9\") pod \"auth-test-model-27be50-kserve-57f66b85bc-bwm45\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") " pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:16.039901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:16.039805    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:16.319764 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:16.319734    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:12:16.321560 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:12:16.321534    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9966c0b_776c_441a_8037_22b82b125ad3.slice/crio-fe4233eaf09e315da068d35d0c73c412e1c2ba634afe5d59b939d97ea4ab7715 WatchSource:0}: Error finding container fe4233eaf09e315da068d35d0c73c412e1c2ba634afe5d59b939d97ea4ab7715: Status 404 returned error can't find the container with id fe4233eaf09e315da068d35d0c73c412e1c2ba634afe5d59b939d97ea4ab7715
Sep 14 05:12:17.150623 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:17.150586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" event={"ID":"d9966c0b-776c-441a-8037-22b82b125ad3","Type":"ContainerStarted","Data":"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"}
Sep 14 05:12:17.150623 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:17.150632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" event={"ID":"d9966c0b-776c-441a-8037-22b82b125ad3","Type":"ContainerStarted","Data":"fe4233eaf09e315da068d35d0c73c412e1c2ba634afe5d59b939d97ea4ab7715"}
Sep 14 05:12:17.169802 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:17.169753    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" podStartSLOduration=2.169735771 podStartE2EDuration="2.169735771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:12:17.16708246 +0000 UTC m=+1531.529034903" watchObservedRunningTime="2026-09-14 05:12:17.169735771 +0000 UTC m=+1531.531688215"
Sep 14 05:12:19.851504 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:19.851469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:19.951327 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:19.951288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:20.615250 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:20.615210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:26.040644 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.040590    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:26.040644 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.040660    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:26.053604 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.053578    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:26.213197 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.213164    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:12:26.618333 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.618298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:26.674634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.674593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:26.753810 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.753771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:26.853280 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:26.853243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:37.378590 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:37.378548    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:37.632503 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:37.632410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:37.769983 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:37.769914    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:12:46.392714 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:46.392680    2571 scope.go:117] "RemoveContainer" containerID="ee59d2c6542718428544d18d6ebd3034de457e997ffcf2fe74fdf01d34f5d450"
Sep 14 05:12:59.410699 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:12:59.410648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:04.825827 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:04.825789    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:04.925873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:04.925829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:05.216291 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:05.216255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:05.504426 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:05.504338    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:05.842599 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:05.842501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:06.313538 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:06.313500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:06.638873 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:06.638787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:07.586963 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:07.586897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:08.285268 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:08.285215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:08.378553 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:08.378518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:09.267695 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:09.267663    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:10.141257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:10.141203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:10.295483 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:10.295452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:10.396823 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:10.396722    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:10.827548 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:10.827482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:11.379447 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.379412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:11.786652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.786621    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:13:11.790684 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.790660    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:11.793206 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.793184    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6cb472-serving-cert\""
Sep 14 05:13:11.802713 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.802688    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:13:11.818739 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.818703    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:11.912400 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.912356    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:11.912400 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.912400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pdmh\" (UniqueName: \"kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:11.912660 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.912476    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:11.912660 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.912524    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:11.914731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:11.914701    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:12.013422 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.013377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.013637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.013426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pdmh\" (UniqueName: \"kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.013637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.013471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.013637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.013531    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.014551 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.014517    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.014769 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.014745    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:12.016310 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.016286    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.016422 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.016357    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.021181 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.021158    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pdmh\" (UniqueName: \"kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh\") pod \"maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.104648 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.104548    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:12.294461 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.294429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:12.382849 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.382822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:12.384263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.384237    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:13:12.386196 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:13:12.386166    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79bc285b_6abc_4153_8935_fec0bd4d552a.slice/crio-036751dadb23f453ad3347a3db35454b04748a9d191c246212e27582b15a5dda WatchSource:0}: Error finding container 036751dadb23f453ad3347a3db35454b04748a9d191c246212e27582b15a5dda: Status 404 returned error can't find the container with id 036751dadb23f453ad3347a3db35454b04748a9d191c246212e27582b15a5dda
Sep 14 05:13:12.387673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.387655    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 05:13:12.473525 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.473484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" event={"ID":"79bc285b-6abc-4153-8935-fec0bd4d552a","Type":"ContainerStarted","Data":"036751dadb23f453ad3347a3db35454b04748a9d191c246212e27582b15a5dda"}
Sep 14 05:13:12.684203 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:12.684107    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:13.485626 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:13.485584    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" event={"ID":"79bc285b-6abc-4153-8935-fec0bd4d552a","Type":"ContainerStarted","Data":"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"}
Sep 14 05:13:13.486059 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:13.485696    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:13.504456 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:13.504286    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" podStartSLOduration=2.041656429 podStartE2EDuration="2.504264698s" totalImagesPullingTime="462.608269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:13:13.503726806 +0000 UTC m=+1587.865679251" watchObservedRunningTime="2026-09-14 05:13:13.504264698 +0000 UTC m=+1587.866217145"
Sep 14 05:13:13.564578 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:13.564545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:14.249308 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:14.249274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:15.050353 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:15.050314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:15.791428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:15.791397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:16.638889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:16.638853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:19.497461 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:19.497430    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:13:22.221513 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.221478    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:13:22.222346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.222309    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" podUID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" containerName="maas-api" containerID="cri-o://52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b" gracePeriod=30
Sep 14 05:13:22.244615 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.244580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:22.309375 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.309345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:22.408174 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.408143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:22.485793 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.485705    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:13:22.507540 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.507499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:22.522561 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.522521    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca\") pod \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") "
Sep 14 05:13:22.522762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.522603    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") pod \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") "
Sep 14 05:13:22.522762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.522647    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2pbf\" (UniqueName: \"kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf\") pod \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") "
Sep 14 05:13:22.522762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.522691    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs\") pod \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\" (UID: \"5083fffa-b380-4ddb-a3b2-e6e46e51d965\") "
Sep 14 05:13:22.522919 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.522858    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5083fffa-b380-4ddb-a3b2-e6e46e51d965" (UID: "5083fffa-b380-4ddb-a3b2-e6e46e51d965"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:13:22.523128 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.523106    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5083fffa-b380-4ddb-a3b2-e6e46e51d965-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:13:22.525275 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.525245    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5083fffa-b380-4ddb-a3b2-e6e46e51d965" (UID: "5083fffa-b380-4ddb-a3b2-e6e46e51d965"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:13:22.525579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.525554    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5083fffa-b380-4ddb-a3b2-e6e46e51d965" (UID: "5083fffa-b380-4ddb-a3b2-e6e46e51d965"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:13:22.525674 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.525553    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf" (OuterVolumeSpecName: "kube-api-access-j2pbf") pod "5083fffa-b380-4ddb-a3b2-e6e46e51d965" (UID: "5083fffa-b380-4ddb-a3b2-e6e46e51d965"). InnerVolumeSpecName "kube-api-access-j2pbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:13:22.546120 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.546087    2571 generic.go:356] "Generic (PLEG): container finished" podID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" containerID="52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b" exitCode=0
Sep 14 05:13:22.546293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.546144    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"
Sep 14 05:13:22.546293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.546169    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" event={"ID":"5083fffa-b380-4ddb-a3b2-e6e46e51d965","Type":"ContainerDied","Data":"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"}
Sep 14 05:13:22.546293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.546234    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v" event={"ID":"5083fffa-b380-4ddb-a3b2-e6e46e51d965","Type":"ContainerDied","Data":"6382a45cad0117f445e92f6d2c83a2ae7d12f358024d63d2f8e1339916074953"}
Sep 14 05:13:22.546293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.546267    2571 scope.go:117] "RemoveContainer" containerID="52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"
Sep 14 05:13:22.558225 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.558198    2571 scope.go:117] "RemoveContainer" containerID="52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"
Sep 14 05:13:22.558652 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:13:22.558619    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b\": container with ID starting with 52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b not found: ID does not exist" containerID="52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"
Sep 14 05:13:22.558766 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.558665    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b"} err="failed to get container status \"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b\": rpc error: code = NotFound desc = could not find container \"52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b\": container with ID starting with 52864fd7e18482c3b59195807b5f59fc6a5d9453e59933d7148889dd00cc7d6b not found: ID does not exist"
Sep 14 05:13:22.569151 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.569120    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:13:22.573109 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.573075    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e08402-6d5bc4bb64-bfb7v"]
Sep 14 05:13:22.624087 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.624057    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:13:22.624087 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.624085    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2pbf\" (UniqueName: \"kubernetes.io/projected/5083fffa-b380-4ddb-a3b2-e6e46e51d965-kube-api-access-j2pbf\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:13:22.624297 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.624098    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5083fffa-b380-4ddb-a3b2-e6e46e51d965-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:13:22.812948 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.812830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:22.913122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:22.913084    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:23.748623 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:23.748588    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:24.195036 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:24.194998    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" path="/var/lib/kubelet/pods/5083fffa-b380-4ddb-a3b2-e6e46e51d965/volumes"
Sep 14 05:13:24.658789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:24.658741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:25.346619 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:25.346573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:26.017574 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:26.017540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:26.922275 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:26.922236    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:27.897465 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:27.897426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:28.112115 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:28.112078    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:28.479394 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:28.479358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:28.605009 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:28.604974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:28.918968 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:28.918784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:29.229028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:29.228997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:29.359427 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:29.359395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:29.865315 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:29.865276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:29.966367 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:29.966335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:30.089145 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:30.089097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:30.184816 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:30.184731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:30.943340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:30.943300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:31.945446 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:31.945411    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:32.371377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:32.371334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:32.464843 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:32.464784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:32.771738 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:32.771701    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:33.568752 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:33.568708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:34.494399 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:34.494364    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:35.264234 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:35.264200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:35.961193 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:35.961162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:36.633240 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.633205    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:13:36.633726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.633645    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" containerName="maas-api"
Sep 14 05:13:36.633726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.633657    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" containerName="maas-api"
Sep 14 05:13:36.633726 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.633720    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5083fffa-b380-4ddb-a3b2-e6e46e51d965" containerName="maas-api"
Sep 14 05:13:36.638395 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.638373    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.640760 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.640736    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-09203c-serving-cert\""
Sep 14 05:13:36.646499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.646475    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:13:36.659764 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.659718    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:36.718362 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.718276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:36.770281 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.770229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.770469 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.770301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vpww\" (UniqueName: \"kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.770469 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.770418    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.770582 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.770501    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.819021 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.818983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:36.871641 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.871590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vpww\" (UniqueName: \"kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.871852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.871680    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.871852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.871778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.872027 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.871869    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.872520 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.872477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.875173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.875151    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.875283 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.875196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.879152 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.879128    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vpww\" (UniqueName: \"kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww\") pod \"maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:36.951697 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:36.951671    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:37.123884 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:37.123835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:37.236794 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:37.236763    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:13:37.238278 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:37.238250    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:37.239832 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:13:37.239802    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1bee7_7595_4c3e_950a_727c032db4f7.slice/crio-6cdbdfe6433a3275de8ede5a9e5e1b468d9086f46866023b6b949df3128170ff WatchSource:0}: Error finding container 6cdbdfe6433a3275de8ede5a9e5e1b468d9086f46866023b6b949df3128170ff: Status 404 returned error can't find the container with id 6cdbdfe6433a3275de8ede5a9e5e1b468d9086f46866023b6b949df3128170ff
Sep 14 05:13:37.637489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:37.637455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" event={"ID":"33c1bee7-7595-4c3e-950a-727c032db4f7","Type":"ContainerStarted","Data":"6cdbdfe6433a3275de8ede5a9e5e1b468d9086f46866023b6b949df3128170ff"}
Sep 14 05:13:38.644541 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:38.644490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" event={"ID":"33c1bee7-7595-4c3e-950a-727c032db4f7","Type":"ContainerStarted","Data":"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"}
Sep 14 05:13:38.645008 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:38.644577    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:38.662247 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:38.662198    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" podStartSLOduration=2.280232902 podStartE2EDuration="2.662184255s" totalImagesPullingTime="381.951353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:13:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:13:38.660858679 +0000 UTC m=+1613.022811119" watchObservedRunningTime="2026-09-14 05:13:38.662184255 +0000 UTC m=+1613.024136698"
Sep 14 05:13:44.656110 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:44.656077    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:13:48.275170 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:48.274070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:48.325695 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:48.325658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:49.000094 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:49.000053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:54.885082 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:54.885046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:58.654374 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:58.654335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:13:59.345092 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:13:59.345047    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:00.072804 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:00.072769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:00.723433 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:00.723383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:01.674447 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:01.674410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:02.345542 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:02.345506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:02.507512 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:02.507473    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:03.143591 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:03.143557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:05.528851 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:05.528811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:06.903569 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:06.903529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:07.254255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:07.254214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:07.950404 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:07.950367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:08.727780 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:08.727748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:09.583232 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:09.583189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:10.264399 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:10.264366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:11.145722 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:11.145684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:11.856510 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:11.856469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:12.670872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:12.670843    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:12.725817 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:12.725787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:15.928423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:15.928391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:21.490202 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.490169    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:14:21.490690 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.490478    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" podUID="79bc285b-6abc-4153-8935-fec0bd4d552a" containerName="maas-api" containerID="cri-o://397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9" gracePeriod=30
Sep 14 05:14:21.518834 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.518797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:21.617771 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.617739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:21.719450 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.719412    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:21.754085 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.754017    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:14:21.818194 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.818156    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:21.900393 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.900345    2571 generic.go:356] "Generic (PLEG): container finished" podID="79bc285b-6abc-4153-8935-fec0bd4d552a" containerID="397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9" exitCode=0
Sep 14 05:14:21.900573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.900400    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"
Sep 14 05:14:21.900573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.900423    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" event={"ID":"79bc285b-6abc-4153-8935-fec0bd4d552a","Type":"ContainerDied","Data":"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"}
Sep 14 05:14:21.900573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.900487    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss" event={"ID":"79bc285b-6abc-4153-8935-fec0bd4d552a","Type":"ContainerDied","Data":"036751dadb23f453ad3347a3db35454b04748a9d191c246212e27582b15a5dda"}
Sep 14 05:14:21.900573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.900513    2571 scope.go:117] "RemoveContainer" containerID="397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"
Sep 14 05:14:21.910350 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.910328    2571 scope.go:117] "RemoveContainer" containerID="397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"
Sep 14 05:14:21.910720 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:14:21.910697    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9\": container with ID starting with 397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9 not found: ID does not exist" containerID="397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"
Sep 14 05:14:21.910833 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.910727    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9"} err="failed to get container status \"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9\": rpc error: code = NotFound desc = could not find container \"397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9\": container with ID starting with 397ed8c49bf39bc603e3fcbdf2e5a6039facd4425d1451310e3b26ff0d8220f9 not found: ID does not exist"
Sep 14 05:14:21.929240 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.929200    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca\") pod \"79bc285b-6abc-4153-8935-fec0bd4d552a\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") "
Sep 14 05:14:21.929446 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.929254    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pdmh\" (UniqueName: \"kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh\") pod \"79bc285b-6abc-4153-8935-fec0bd4d552a\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") "
Sep 14 05:14:21.929446 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.929303    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls\") pod \"79bc285b-6abc-4153-8935-fec0bd4d552a\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") "
Sep 14 05:14:21.929446 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.929320    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs\") pod \"79bc285b-6abc-4153-8935-fec0bd4d552a\" (UID: \"79bc285b-6abc-4153-8935-fec0bd4d552a\") "
Sep 14 05:14:21.929664 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.929636    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79bc285b-6abc-4153-8935-fec0bd4d552a" (UID: "79bc285b-6abc-4153-8935-fec0bd4d552a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:14:21.931723 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.931685    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "79bc285b-6abc-4153-8935-fec0bd4d552a" (UID: "79bc285b-6abc-4153-8935-fec0bd4d552a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:14:21.931723 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.931703    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79bc285b-6abc-4153-8935-fec0bd4d552a" (UID: "79bc285b-6abc-4153-8935-fec0bd4d552a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:14:21.931905 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:21.931731    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh" (OuterVolumeSpecName: "kube-api-access-8pdmh") pod "79bc285b-6abc-4153-8935-fec0bd4d552a" (UID: "79bc285b-6abc-4153-8935-fec0bd4d552a"). InnerVolumeSpecName "kube-api-access-8pdmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:14:22.030639 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.030544    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:22.030639 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.030575    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bc285b-6abc-4153-8935-fec0bd4d552a-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:22.030639 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.030587    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bc285b-6abc-4153-8935-fec0bd4d552a-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:22.030639 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.030595    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pdmh\" (UniqueName: \"kubernetes.io/projected/79bc285b-6abc-4153-8935-fec0bd4d552a-kube-api-access-8pdmh\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:22.130339 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.130305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:22.182246 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.182210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:22.221729 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.221688    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:14:22.227172 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.227140    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6cb472-7c88bb6db5-jg9ss"]
Sep 14 05:14:22.854132 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:22.854103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:23.722349 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:23.722318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:24.195328 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:24.195291    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79bc285b-6abc-4153-8935-fec0bd4d552a" path="/var/lib/kubelet/pods/79bc285b-6abc-4153-8935-fec0bd4d552a/volumes"
Sep 14 05:14:24.573680 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:24.573644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:25.250308 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:25.250275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:25.928071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:25.928031    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:28.199259 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:28.199223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:28.260118 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:28.260084    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:28.465471 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:28.465439    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:28.974872 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:28.974838    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:29.759040 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:29.758990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:30.531874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:30.531823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:31.283152 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:31.283119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:31.957138 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:31.957097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:32.786585 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:32.786542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:33.579314 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:33.579283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.285373 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.285331    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:14:44.286364 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.286325    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" podUID="33c1bee7-7595-4c3e-950a-727c032db4f7" containerName="maas-api" containerID="cri-o://5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08" gracePeriod=30
Sep 14 05:14:44.309895 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.309863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.380891 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.380846    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.435528 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.435487    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.506101 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.506056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.552100 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.552017    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:14:44.560504 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.560476    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:44.647260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647217    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca\") pod \"33c1bee7-7595-4c3e-950a-727c032db4f7\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") "
Sep 14 05:14:44.647436 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647317    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls\") pod \"33c1bee7-7595-4c3e-950a-727c032db4f7\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") "
Sep 14 05:14:44.647436 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647399    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vpww\" (UniqueName: \"kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww\") pod \"33c1bee7-7595-4c3e-950a-727c032db4f7\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") "
Sep 14 05:14:44.647520 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647476    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs\") pod \"33c1bee7-7595-4c3e-950a-727c032db4f7\" (UID: \"33c1bee7-7595-4c3e-950a-727c032db4f7\") "
Sep 14 05:14:44.647691 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647669    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33c1bee7-7595-4c3e-950a-727c032db4f7" (UID: "33c1bee7-7595-4c3e-950a-727c032db4f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:14:44.647804 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.647788    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1bee7-7595-4c3e-950a-727c032db4f7-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:44.649774 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.649740    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33c1bee7-7595-4c3e-950a-727c032db4f7" (UID: "33c1bee7-7595-4c3e-950a-727c032db4f7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:14:44.649885 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.649751    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33c1bee7-7595-4c3e-950a-727c032db4f7" (UID: "33c1bee7-7595-4c3e-950a-727c032db4f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:14:44.649885 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.649832    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww" (OuterVolumeSpecName: "kube-api-access-6vpww") pod "33c1bee7-7595-4c3e-950a-727c032db4f7" (UID: "33c1bee7-7595-4c3e-950a-727c032db4f7"). InnerVolumeSpecName "kube-api-access-6vpww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:14:44.749129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.749089    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vpww\" (UniqueName: \"kubernetes.io/projected/33c1bee7-7595-4c3e-950a-727c032db4f7-kube-api-access-6vpww\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:44.749129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.749121    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:44.749129 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.749130    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c1bee7-7595-4c3e-950a-727c032db4f7-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:14:44.905260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:44.905166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:45.005305 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.005268    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:45.043833 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.043796    2571 generic.go:356] "Generic (PLEG): container finished" podID="33c1bee7-7595-4c3e-950a-727c032db4f7" containerID="5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08" exitCode=0
Sep 14 05:14:45.044051 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.043848    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"
Sep 14 05:14:45.044051 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.043885    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" event={"ID":"33c1bee7-7595-4c3e-950a-727c032db4f7","Type":"ContainerDied","Data":"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"}
Sep 14 05:14:45.044051 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.043977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr" event={"ID":"33c1bee7-7595-4c3e-950a-727c032db4f7","Type":"ContainerDied","Data":"6cdbdfe6433a3275de8ede5a9e5e1b468d9086f46866023b6b949df3128170ff"}
Sep 14 05:14:45.044051 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.044011    2571 scope.go:117] "RemoveContainer" containerID="5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"
Sep 14 05:14:45.054037 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.054017    2571 scope.go:117] "RemoveContainer" containerID="5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"
Sep 14 05:14:45.054356 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:14:45.054338    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08\": container with ID starting with 5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08 not found: ID does not exist" containerID="5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"
Sep 14 05:14:45.054406 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.054366    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08"} err="failed to get container status \"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08\": rpc error: code = NotFound desc = could not find container \"5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08\": container with ID starting with 5d0aab484eec4206e52f61fd6e036398472f39b1922adee1266d875f7030de08 not found: ID does not exist"
Sep 14 05:14:45.064776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.064738    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:14:45.069319 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.069286    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-09203c-64594dbfd9-hgdzr"]
Sep 14 05:14:45.753394 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:45.753364    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:46.197009 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:46.196906    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c1bee7-7595-4c3e-950a-727c032db4f7" path="/var/lib/kubelet/pods/33c1bee7-7595-4c3e-950a-727c032db4f7/volumes"
Sep 14 05:14:46.661852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:46.661814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:47.352508 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:47.352478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:48.022822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:48.022785    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:50.733134 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:50.733104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:50.788700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:50.788662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:51.015071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:51.014977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:51.571443 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:51.571403    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:52.034148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:52.034116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:52.529909 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:52.529873    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:53.150244 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:53.150211    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:14:53.171343 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:53.171305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:53.466692 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:53.466657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:54.156757 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:54.156727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:55.072957 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:55.072890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:55.758886 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:55.758852    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:14:58.840298 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:14:58.840266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.050736 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.050695    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:15:02.051317 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.050970    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" podUID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" containerName="maas-api" containerID="cri-o://93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6" gracePeriod=30
Sep 14 05:15:02.072106 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.069481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.169423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.169365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.316486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.316416    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:15:02.322038 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.322004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.385285 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.385249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.422377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.422347    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs\") pod \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") "
Sep 14 05:15:02.422568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.422404    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls\") pod \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") "
Sep 14 05:15:02.422568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.422447    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca\") pod \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") "
Sep 14 05:15:02.422568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.422495    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2nr7\" (UniqueName: \"kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7\") pod \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\" (UID: \"6b785fa4-8517-4aa9-8d28-37c32e73e13b\") "
Sep 14 05:15:02.422862 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.422832    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b785fa4-8517-4aa9-8d28-37c32e73e13b" (UID: "6b785fa4-8517-4aa9-8d28-37c32e73e13b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:15:02.424878 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.424848    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b785fa4-8517-4aa9-8d28-37c32e73e13b" (UID: "6b785fa4-8517-4aa9-8d28-37c32e73e13b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:15:02.424878 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.424867    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7" (OuterVolumeSpecName: "kube-api-access-q2nr7") pod "6b785fa4-8517-4aa9-8d28-37c32e73e13b" (UID: "6b785fa4-8517-4aa9-8d28-37c32e73e13b"). InnerVolumeSpecName "kube-api-access-q2nr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:15:02.425060 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.424905    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b785fa4-8517-4aa9-8d28-37c32e73e13b" (UID: "6b785fa4-8517-4aa9-8d28-37c32e73e13b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:15:02.523540 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.523491    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:02.523540 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.523535    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b785fa4-8517-4aa9-8d28-37c32e73e13b-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:02.523540 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.523548    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b785fa4-8517-4aa9-8d28-37c32e73e13b-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:02.523804 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.523562    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2nr7\" (UniqueName: \"kubernetes.io/projected/6b785fa4-8517-4aa9-8d28-37c32e73e13b-kube-api-access-q2nr7\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:02.763095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.763060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:02.863613 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:02.863577    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:03.149232 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.149130    2571 generic.go:356] "Generic (PLEG): container finished" podID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" containerID="93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6" exitCode=0
Sep 14 05:15:03.149232 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.149183    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"
Sep 14 05:15:03.149746 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.149227    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" event={"ID":"6b785fa4-8517-4aa9-8d28-37c32e73e13b","Type":"ContainerDied","Data":"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"}
Sep 14 05:15:03.149746 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.149286    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949" event={"ID":"6b785fa4-8517-4aa9-8d28-37c32e73e13b","Type":"ContainerDied","Data":"c666a067ed4745006dc1bc2c93dfbcffca756e1b7163d9db94b038063bfbe10e"}
Sep 14 05:15:03.149746 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.149310    2571 scope.go:117] "RemoveContainer" containerID="93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"
Sep 14 05:15:03.159866 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.159844    2571 scope.go:117] "RemoveContainer" containerID="93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"
Sep 14 05:15:03.160310 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:15:03.160282    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6\": container with ID starting with 93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6 not found: ID does not exist" containerID="93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"
Sep 14 05:15:03.160410 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.160324    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6"} err="failed to get container status \"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6\": rpc error: code = NotFound desc = could not find container \"93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6\": container with ID starting with 93264c8eecfc37ae456c023b7a45bf2ea937062b5669e99d0e2d6817d84a0ad6 not found: ID does not exist"
Sep 14 05:15:03.178612 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.178578    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:15:03.188016 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.187975    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-2eeb50-5db48c9bbf-gp949"]
Sep 14 05:15:03.468851 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:03.468818    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:04.196313 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:04.196273    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" path="/var/lib/kubelet/pods/6b785fa4-8517-4aa9-8d28-37c32e73e13b/volumes"
Sep 14 05:15:04.423974 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:04.423913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:05.027993 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:05.027952    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:08.172888 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.172788    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" podUID="d9966c0b-776c-441a-8037-22b82b125ad3" containerName="main" containerID="cri-o://a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb" gracePeriod=45
Sep 14 05:15:08.429762 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.429690    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:15:08.581439 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581403    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.581694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581515    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.581694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581533    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.581694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581566    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwpp9\" (UniqueName: \"kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.581694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581600    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.581694 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581638    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir\") pod \"d9966c0b-776c-441a-8037-22b82b125ad3\" (UID: \"d9966c0b-776c-441a-8037-22b82b125ad3\") "
Sep 14 05:15:08.582028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581733    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home" (OuterVolumeSpecName: "home") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:15:08.582028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581878    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:15:08.582028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.581970    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:15:08.582028 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.582003    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-home\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.584094 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.584067    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:15:08.584247 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.584178    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9" (OuterVolumeSpecName: "kube-api-access-xwpp9") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "kube-api-access-xwpp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:15:08.584307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.584294    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm" (OuterVolumeSpecName: "dshm") pod "d9966c0b-776c-441a-8037-22b82b125ad3" (UID: "d9966c0b-776c-441a-8037-22b82b125ad3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:15:08.683043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.682910    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwpp9\" (UniqueName: \"kubernetes.io/projected/d9966c0b-776c-441a-8037-22b82b125ad3-kube-api-access-xwpp9\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.683043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.682973    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-dshm\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.683043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.682983    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-tmp-dir\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.683043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.682992    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9966c0b-776c-441a-8037-22b82b125ad3-tls-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.683043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.683001    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9966c0b-776c-441a-8037-22b82b125ad3-model-cache\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:08.896414 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.896374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:08.980324 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:08.980293    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:09.195201 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.195169    2571 generic.go:356] "Generic (PLEG): container finished" podID="d9966c0b-776c-441a-8037-22b82b125ad3" containerID="a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb" exitCode=0
Sep 14 05:15:09.195634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.195210    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"
Sep 14 05:15:09.195634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.195252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" event={"ID":"d9966c0b-776c-441a-8037-22b82b125ad3","Type":"ContainerDied","Data":"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"}
Sep 14 05:15:09.195634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.195296    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45" event={"ID":"d9966c0b-776c-441a-8037-22b82b125ad3","Type":"ContainerDied","Data":"fe4233eaf09e315da068d35d0c73c412e1c2ba634afe5d59b939d97ea4ab7715"}
Sep 14 05:15:09.195634 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.195318    2571 scope.go:117] "RemoveContainer" containerID="a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"
Sep 14 05:15:09.198350 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.198304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:09.205583 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.205565    2571 scope.go:117] "RemoveContainer" containerID="a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"
Sep 14 05:15:09.205919 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:15:09.205900    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb\": container with ID starting with a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb not found: ID does not exist" containerID="a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"
Sep 14 05:15:09.206035 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.205952    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb"} err="failed to get container status \"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb\": rpc error: code = NotFound desc = could not find container \"a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb\": container with ID starting with a215bd9bf029ccabc25dfd8b097db4621514bbcaac8deb12a5a78869ba8f52bb not found: ID does not exist"
Sep 14 05:15:09.217699 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.217659    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:15:09.221943 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.221894    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-27be50/auth-test-model-27be50-kserve-57f66b85bc-bwm45"]
Sep 14 05:15:09.857423 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:09.857383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:10.194681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:10.194596    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9966c0b-776c-441a-8037-22b82b125ad3" path="/var/lib/kubelet/pods/d9966c0b-776c-441a-8037-22b82b125ad3/volumes"
Sep 14 05:15:10.759299 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:10.759263    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:11.474509 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:11.474471    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:12.247122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:12.247092    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:12.990811 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:12.990774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.070501 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.070464    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:15:20.072310 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.072269    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" podUID="6d21b65d-529b-4849-ab0f-e47da120bfcf" containerName="maas-api" containerID="cri-o://23efe3aac1c5530103f2d91a3f9a1642369723c300036da6e9b0e9477672c55a" gracePeriod=30
Sep 14 05:15:20.100342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.100298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.200266 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.200197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.259363 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.259332    2571 generic.go:356] "Generic (PLEG): container finished" podID="6d21b65d-529b-4849-ab0f-e47da120bfcf" containerID="23efe3aac1c5530103f2d91a3f9a1642369723c300036da6e9b0e9477672c55a" exitCode=0
Sep 14 05:15:20.259523 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.259401    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" event={"ID":"6d21b65d-529b-4849-ab0f-e47da120bfcf","Type":"ContainerDied","Data":"23efe3aac1c5530103f2d91a3f9a1642369723c300036da6e9b0e9477672c55a"}
Sep 14 05:15:20.261969 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.261905    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.321167 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.321084    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.331480 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.331452    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:15:20.397479 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.397441    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca\") pod \"6d21b65d-529b-4849-ab0f-e47da120bfcf\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") "
Sep 14 05:15:20.397664 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.397530    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6gdn\" (UniqueName: \"kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn\") pod \"6d21b65d-529b-4849-ab0f-e47da120bfcf\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") "
Sep 14 05:15:20.397664 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.397551    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs\") pod \"6d21b65d-529b-4849-ab0f-e47da120bfcf\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") "
Sep 14 05:15:20.397664 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.397625    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls\") pod \"6d21b65d-529b-4849-ab0f-e47da120bfcf\" (UID: \"6d21b65d-529b-4849-ab0f-e47da120bfcf\") "
Sep 14 05:15:20.397990 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.397922    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d21b65d-529b-4849-ab0f-e47da120bfcf" (UID: "6d21b65d-529b-4849-ab0f-e47da120bfcf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:15:20.399985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.399923    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6d21b65d-529b-4849-ab0f-e47da120bfcf" (UID: "6d21b65d-529b-4849-ab0f-e47da120bfcf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:15:20.399985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.399971    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn" (OuterVolumeSpecName: "kube-api-access-f6gdn") pod "6d21b65d-529b-4849-ab0f-e47da120bfcf" (UID: "6d21b65d-529b-4849-ab0f-e47da120bfcf"). InnerVolumeSpecName "kube-api-access-f6gdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:15:20.400131 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.400022    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d21b65d-529b-4849-ab0f-e47da120bfcf" (UID: "6d21b65d-529b-4849-ab0f-e47da120bfcf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:15:20.498693 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.498657    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:20.498693 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.498693    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d21b65d-529b-4849-ab0f-e47da120bfcf-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:20.498693 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.498707    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6gdn\" (UniqueName: \"kubernetes.io/projected/6d21b65d-529b-4849-ab0f-e47da120bfcf-kube-api-access-f6gdn\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:20.498976 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.498719    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d21b65d-529b-4849-ab0f-e47da120bfcf-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:15:20.711995 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.711952    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:20.810700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:20.810652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:21.266409 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.266377    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"
Sep 14 05:15:21.266409 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.266395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv" event={"ID":"6d21b65d-529b-4849-ab0f-e47da120bfcf","Type":"ContainerDied","Data":"1ac5587239dcf78d1737cf0ea6b340b82e337c49038bc9c266458ca9c5f487b4"}
Sep 14 05:15:21.266986 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.266455    2571 scope.go:117] "RemoveContainer" containerID="23efe3aac1c5530103f2d91a3f9a1642369723c300036da6e9b0e9477672c55a"
Sep 14 05:15:21.289795 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.289766    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:15:21.293733 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.293710    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-a8c46f-7c94f68bfc-ksmlv"]
Sep 14 05:15:21.457247 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:21.457215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:22.195251 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:22.195208    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d21b65d-529b-4849-ab0f-e47da120bfcf" path="/var/lib/kubelet/pods/6d21b65d-529b-4849-ab0f-e47da120bfcf/volumes"
Sep 14 05:15:22.231308 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:22.231270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:26.528249 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:26.528192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:26.591296 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:26.591258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:26.791749 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:26.791637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:34.217974 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:34.217892    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:15:50.762954 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:15:50.762894    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:16:08.504148 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504113    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7536-b89fd3"]
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504553    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c1bee7-7595-4c3e-950a-727c032db4f7" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504566    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c1bee7-7595-4c3e-950a-727c032db4f7" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504586    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504592    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504603    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d21b65d-529b-4849-ab0f-e47da120bfcf" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504610    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d21b65d-529b-4849-ab0f-e47da120bfcf" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504617    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9966c0b-776c-441a-8037-22b82b125ad3" containerName="main"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504622    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9966c0b-776c-441a-8037-22b82b125ad3" containerName="main"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504631    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79bc285b-6abc-4153-8935-fec0bd4d552a" containerName="maas-api"
Sep 14 05:16:08.504642 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504636    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79bc285b-6abc-4153-8935-fec0bd4d552a" containerName="maas-api"
Sep 14 05:16:08.505024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504696    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d21b65d-529b-4849-ab0f-e47da120bfcf" containerName="maas-api"
Sep 14 05:16:08.505024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504703    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c1bee7-7595-4c3e-950a-727c032db4f7" containerName="maas-api"
Sep 14 05:16:08.505024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504711    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9966c0b-776c-441a-8037-22b82b125ad3" containerName="main"
Sep 14 05:16:08.505024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504716    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79bc285b-6abc-4153-8935-fec0bd4d552a" containerName="maas-api"
Sep 14 05:16:08.505024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.504722    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b785fa4-8517-4aa9-8d28-37c32e73e13b" containerName="maas-api"
Sep 14 05:16:08.509130 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.509112    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:08.511693 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.511672    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dcz7p\""
Sep 14 05:16:08.515146 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.515114    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7536-b89fd3"]
Sep 14 05:16:08.540618 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.540584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pqxh\" (UniqueName: \"kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh\") pod \"test-curl-7536-b89fd3\" (UID: \"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5\") " pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:08.641754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.641714    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pqxh\" (UniqueName: \"kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh\") pod \"test-curl-7536-b89fd3\" (UID: \"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5\") " pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:08.651641 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.651614    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pqxh\" (UniqueName: \"kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh\") pod \"test-curl-7536-b89fd3\" (UID: \"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5\") " pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:08.820849 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:08.820756    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:09.099603 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:09.099575    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7536-b89fd3"]
Sep 14 05:16:09.102067 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:16:09.102039    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda384adaa_8ba0_4ce7_9c5b_ca3592dc36f5.slice/crio-6a43dbf28bee7c0e41004b2ff5993a0e6edbf1ad2ccc608a2f386c79c37b1fa7 WatchSource:0}: Error finding container 6a43dbf28bee7c0e41004b2ff5993a0e6edbf1ad2ccc608a2f386c79c37b1fa7: Status 404 returned error can't find the container with id 6a43dbf28bee7c0e41004b2ff5993a0e6edbf1ad2ccc608a2f386c79c37b1fa7
Sep 14 05:16:09.530763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:09.530713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7536-b89fd3" event={"ID":"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5","Type":"ContainerStarted","Data":"d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a"}
Sep 14 05:16:09.530763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:09.530765    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7536-b89fd3" event={"ID":"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5","Type":"ContainerStarted","Data":"6a43dbf28bee7c0e41004b2ff5993a0e6edbf1ad2ccc608a2f386c79c37b1fa7"}
Sep 14 05:16:09.549274 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:09.549189    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7536-b89fd3" podStartSLOduration=1.5491717820000002 podStartE2EDuration="1.549171782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:16:09.548191054 +0000 UTC m=+1763.910143497" watchObservedRunningTime="2026-09-14 05:16:09.549171782 +0000 UTC m=+1763.911124225"
Sep 14 05:16:10.220011 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.219969    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7536-b89fd3"]
Sep 14 05:16:10.225346 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.225309    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7536-b89fd3"]
Sep 14 05:16:10.370115 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.370028    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7536-e0e519"]
Sep 14 05:16:10.370473 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.370459    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" containerName="test-curl-7536-b89fd3"
Sep 14 05:16:10.370518 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.370474    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" containerName="test-curl-7536-b89fd3"
Sep 14 05:16:10.370560 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.370534    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" containerName="test-curl-7536-b89fd3"
Sep 14 05:16:10.373853 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.373835    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:10.380248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.380219    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7536-e0e519"]
Sep 14 05:16:10.458772 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.458733    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llwdf\" (UniqueName: \"kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf\") pod \"test-curl-7536-e0e519\" (UID: \"0d86a048-5dc3-4909-8990-8a41de7e68b8\") " pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:10.535516 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:10.535487    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7536-b89fd3" container="test-curl-7536-b89fd3"
Sep 14 05:16:10.560047 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.560009    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llwdf\" (UniqueName: \"kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf\") pod \"test-curl-7536-e0e519\" (UID: \"0d86a048-5dc3-4909-8990-8a41de7e68b8\") " pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:10.570822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.570782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llwdf\" (UniqueName: \"kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf\") pod \"test-curl-7536-e0e519\" (UID: \"0d86a048-5dc3-4909-8990-8a41de7e68b8\") " pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:10.684441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.684356    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:10.964576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:10.964543    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7536-e0e519"]
Sep 14 05:16:10.967148 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:16:10.967112    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d86a048_5dc3_4909_8990_8a41de7e68b8.slice/crio-6d7d18a6bb497a519dc8cf431c43c1491daa7f959762ec259a0ebe5a9c557e22 WatchSource:0}: Error finding container 6d7d18a6bb497a519dc8cf431c43c1491daa7f959762ec259a0ebe5a9c557e22: Status 404 returned error can't find the container with id 6d7d18a6bb497a519dc8cf431c43c1491daa7f959762ec259a0ebe5a9c557e22
Sep 14 05:16:11.541763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:11.541728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7536-e0e519" event={"ID":"0d86a048-5dc3-4909-8990-8a41de7e68b8","Type":"ContainerStarted","Data":"0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa"}
Sep 14 05:16:11.541763 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:11.541771    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7536-e0e519" event={"ID":"0d86a048-5dc3-4909-8990-8a41de7e68b8","Type":"ContainerStarted","Data":"6d7d18a6bb497a519dc8cf431c43c1491daa7f959762ec259a0ebe5a9c557e22"}
Sep 14 05:16:11.542279 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:11.541790    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7536-b89fd3" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" containerName="test-curl-7536-b89fd3" containerID="cri-o://d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a" gracePeriod=2
Sep 14 05:16:11.544958 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:11.544911    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7536-b89fd3\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:11.566057 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:11.566000    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7536-e0e519" podStartSLOduration=1.565982465 podStartE2EDuration="1.565982465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:16:11.565411971 +0000 UTC m=+1765.927364413" watchObservedRunningTime="2026-09-14 05:16:11.565982465 +0000 UTC m=+1765.927934941"
Sep 14 05:16:11.567911 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:11.567884    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7536-b89fd3\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:12.184907 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:12.184866    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7536-e0e519"]
Sep 14 05:16:12.196687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:12.196653    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7536-e0e519"]
Sep 14 05:16:12.547426 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:12.547393    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7536-e0e519" container="test-curl-7536-e0e519"
Sep 14 05:16:13.554105 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:13.554067    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7536-e0e519" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" containerName="test-curl-7536-e0e519" containerID="cri-o://0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa" gracePeriod=2
Sep 14 05:16:13.556592 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:13.556551    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7536-e0e519\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:13.697348 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:13.697317    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:13.700050 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:13.700019    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7536-e0e519\" is forbidden: User \"system:node:ip-10-0-137-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-105.ec2.internal' and this object" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:13.788208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:13.788170    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pqxh\" (UniqueName: \"kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh\") pod \"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5\" (UID: \"a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5\") "
Sep 14 05:16:13.790475 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:13.790443    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh" (OuterVolumeSpecName: "kube-api-access-6pqxh") pod "a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" (UID: "a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5"). InnerVolumeSpecName "kube-api-access-6pqxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:16:13.888871 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:13.888834    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pqxh\" (UniqueName: \"kubernetes.io/projected/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5-kube-api-access-6pqxh\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:16:14.196444 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.196406    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" path="/var/lib/kubelet/pods/a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5/volumes"
Sep 14 05:16:14.277630 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.277598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:16:14.560856 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.560815    2571 generic.go:356] "Generic (PLEG): container finished" podID="a384adaa-8ba0-4ce7-9c5b-ca3592dc36f5" containerID="d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a" exitCode=137
Sep 14 05:16:14.560856 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.560859    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-b89fd3"
Sep 14 05:16:14.561371 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.560881    2571 scope.go:117] "RemoveContainer" containerID="d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a"
Sep 14 05:16:14.571161 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.571132    2571 scope.go:117] "RemoveContainer" containerID="d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a"
Sep 14 05:16:14.571625 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:14.571599    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a\": container with ID starting with d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a not found: ID does not exist" containerID="d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a"
Sep 14 05:16:14.571730 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:14.571641    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a"} err="failed to get container status \"d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a\": rpc error: code = NotFound desc = could not find container \"d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a\": container with ID starting with d0bc2589916ed2def87bbfc5efed520c13c13106fe2af486f81f25eaafa58f1a not found: ID does not exist"
Sep 14 05:16:15.370836 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:15.370792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:16:15.719794 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:15.719766    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:15.808029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:15.807989    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llwdf\" (UniqueName: \"kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf\") pod \"0d86a048-5dc3-4909-8990-8a41de7e68b8\" (UID: \"0d86a048-5dc3-4909-8990-8a41de7e68b8\") "
Sep 14 05:16:15.810260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:15.810228    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf" (OuterVolumeSpecName: "kube-api-access-llwdf") pod "0d86a048-5dc3-4909-8990-8a41de7e68b8" (UID: "0d86a048-5dc3-4909-8990-8a41de7e68b8"). InnerVolumeSpecName "kube-api-access-llwdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:16:15.909731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:15.909691    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llwdf\" (UniqueName: \"kubernetes.io/projected/0d86a048-5dc3-4909-8990-8a41de7e68b8-kube-api-access-llwdf\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:16:16.194709 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.194674    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" path="/var/lib/kubelet/pods/0d86a048-5dc3-4909-8990-8a41de7e68b8/volumes"
Sep 14 05:16:16.576049 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.576009    2571 generic.go:356] "Generic (PLEG): container finished" podID="0d86a048-5dc3-4909-8990-8a41de7e68b8" containerID="0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa" exitCode=137
Sep 14 05:16:16.576269 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.576060    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7536-e0e519"
Sep 14 05:16:16.576269 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.576113    2571 scope.go:117] "RemoveContainer" containerID="0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa"
Sep 14 05:16:16.585183 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.585163    2571 scope.go:117] "RemoveContainer" containerID="0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa"
Sep 14 05:16:16.585549 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:16:16.585523    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa\": container with ID starting with 0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa not found: ID does not exist" containerID="0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa"
Sep 14 05:16:16.585654 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:16.585561    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa"} err="failed to get container status \"0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa\": rpc error: code = NotFound desc = could not find container \"0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa\": container with ID starting with 0b9a429ade9308f6afa6b3143bd872868e9afb72886bbe6f4dbed40eb30afdaa not found: ID does not exist"
Sep 14 05:16:19.275794 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:19.275754    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:16:22.588084 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:22.588052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:16:46.261544 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:46.261514    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:16:46.264631 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:46.264608    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:16:46.266846 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:46.266825    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:16:46.270271 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:46.270240    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:16:48.022850 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:16:48.022797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:19.034557 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:19.034518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:22.366569 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:22.366533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:23.428215 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:23.428161    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:23.894128 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:23.894090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:24.226754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:24.226708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:24.660395 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:24.660321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:26.445897 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:26.445859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:27.162773 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:27.162731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:27.244401 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:27.244373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:28.154450 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.154392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:28.926987 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.926949    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:17:28.927637 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.927615    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" containerName="test-curl-7536-e0e519"
Sep 14 05:17:28.927683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.927643    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" containerName="test-curl-7536-e0e519"
Sep 14 05:17:28.927789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.927775    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d86a048-5dc3-4909-8990-8a41de7e68b8" containerName="test-curl-7536-e0e519"
Sep 14 05:17:28.931632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.931606    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:28.933866 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.933842    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-524a38-serving-cert\""
Sep 14 05:17:28.937884 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.937858    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:17:28.960479 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:28.960446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:29.027273 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.027232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.027273 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.027279    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnmr\" (UniqueName: \"kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.027526 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.027347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.027526 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.027406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.061877 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.061847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:29.128529 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.128480    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.128731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.128540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.128731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.128573    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnmr\" (UniqueName: \"kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.128731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.128620    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.128905 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:17:29.128745    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-serving-cert: secret "maas-api-e2e-ipp-a-524a38-serving-cert" not found
Sep 14 05:17:29.128905 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:17:29.128862    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls podName:c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0 nodeName:}" failed. No retries permitted until 2026-09-14 05:17:29.628838571 +0000 UTC m=+1843.990791012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls") pod "maas-api-e2e-ipp-a-524a38-c6644467-x28kf" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0") : secret "maas-api-e2e-ipp-a-524a38-serving-cert" not found
Sep 14 05:17:29.129384 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.129366    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.131484 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.131455    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.137340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.137307    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnmr\" (UniqueName: \"kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.411264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.411219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:29.512816 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.512787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:29.632725 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.632686    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:29.632974 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:17:29.632856    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-serving-cert: secret "maas-api-e2e-ipp-a-524a38-serving-cert" not found
Sep 14 05:17:29.633063 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:17:29.632982    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls podName:c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0 nodeName:}" failed. No retries permitted until 2026-09-14 05:17:30.632960777 +0000 UTC m=+1844.994913213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls") pod "maas-api-e2e-ipp-a-524a38-c6644467-x28kf" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0") : secret "maas-api-e2e-ipp-a-524a38-serving-cert" not found
Sep 14 05:17:29.769796 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:29.769760    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:30.638683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:30.638640    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:30.641281 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:30.641254    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-524a38-c6644467-x28kf\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:30.745493 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:30.745445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:31.025003 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:31.024969    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:17:31.027914 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:17:31.027878    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5db76b2_8dcf_4f8b_a1d0_dd4986c82fc0.slice/crio-8b511d41af5bd250e4a084485fcc7145ab63bd52192d21670ae54e79307e9adb WatchSource:0}: Error finding container 8b511d41af5bd250e4a084485fcc7145ab63bd52192d21670ae54e79307e9adb: Status 404 returned error can't find the container with id 8b511d41af5bd250e4a084485fcc7145ab63bd52192d21670ae54e79307e9adb
Sep 14 05:17:31.997348 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:31.997310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" event={"ID":"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0","Type":"ContainerStarted","Data":"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"}
Sep 14 05:17:31.997740 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:31.997360    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" event={"ID":"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0","Type":"ContainerStarted","Data":"8b511d41af5bd250e4a084485fcc7145ab63bd52192d21670ae54e79307e9adb"}
Sep 14 05:17:31.997740 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:31.997461    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:32.017126 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:32.017066    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" podStartSLOduration=3.509208676 podStartE2EDuration="4.017045388s" totalImagesPullingTime="507.836712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:17:32.015280567 +0000 UTC m=+1846.377233014" watchObservedRunningTime="2026-09-14 05:17:32.017045388 +0000 UTC m=+1846.378997831"
Sep 14 05:17:38.008804 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:38.008773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:17:51.035260 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:51.035224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:51.858782 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:51.858741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:52.254544 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:52.254513    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:52.550607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:52.550525    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:53.036638 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:53.036601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:53.245817 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:53.245782    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:53.353408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:53.353326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:53.900721 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:53.900683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:56.962848 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:56.962810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:57.057624 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:57.057585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:57.768553 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:57.768519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:58.536533 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:58.536490    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.172513 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.172463    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:17:59.177212 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.177180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.179626 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.179597    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-7bbea9-serving-cert\""
Sep 14 05:17:59.188190 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.188160    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:17:59.200456 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.200425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.258049 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.258009    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.306290 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.306242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.306290 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.306290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d525v\" (UniqueName: \"kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.306564 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.306395    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.306564 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.306462    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.407174 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.407130    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.407396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.407193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.407396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.407276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.407396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.407300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d525v\" (UniqueName: \"kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.407908 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.407883    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.410158 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.410125    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.410269 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.410161    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.416421 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.416393    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d525v\" (UniqueName: \"kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v\") pod \"maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.433670 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.433585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.493573 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.493540    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:17:59.680218 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.680162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.773824 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.773665    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:17:59.775280 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:17:59.775247    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:17:59.777270 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:17:59.777235    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c1d955_a942_4f5a_ba38_0792a37411ad.slice/crio-c1b619ed4398db072ce5c773ff235a6094afe0f1813d2709c07d5e80efd91180 WatchSource:0}: Error finding container c1b619ed4398db072ce5c773ff235a6094afe0f1813d2709c07d5e80efd91180: Status 404 returned error can't find the container with id c1b619ed4398db072ce5c773ff235a6094afe0f1813d2709c07d5e80efd91180
Sep 14 05:18:00.148066 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:00.147971    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" event={"ID":"92c1d955-a942-4f5a-ba38-0792a37411ad","Type":"ContainerStarted","Data":"c1b619ed4398db072ce5c773ff235a6094afe0f1813d2709c07d5e80efd91180"}
Sep 14 05:18:01.155722 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:01.155681    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" event={"ID":"92c1d955-a942-4f5a-ba38-0792a37411ad","Type":"ContainerStarted","Data":"bb11563d1b844eeadcae23f8e93ca2b1bb140f01383413fffeadfcad4663b716"}
Sep 14 05:18:01.156171 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:01.155761    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:18:01.174328 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:01.174269    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" podStartSLOduration=1.695031045 podStartE2EDuration="2.174249124s" totalImagesPullingTime="479.218079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:18:01.173323519 +0000 UTC m=+1875.535275963" watchObservedRunningTime="2026-09-14 05:18:01.174249124 +0000 UTC m=+1875.536201567"
Sep 14 05:18:07.166702 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:07.166616    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:18:13.120121 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:13.120083    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:21.257358 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:21.257320    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:22.044592 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:22.044539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:22.145015 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:22.144980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:22.737946 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:22.737877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:23.392977 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:23.392942    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:24.228547 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:24.228507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:18:43.856272 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:18:43.856236    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:05.659751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:05.659714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:09.372122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:09.372069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:10.936654 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:10.936622    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:12.165477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:12.165445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:12.800656 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:12.800617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:12.899958 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:12.899896    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:12.999881 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:12.999842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:13.734180 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:13.734140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:14.631593 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:14.631557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:15.516133 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:15.516100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:16.175663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.175617    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:16.180458 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.180429    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.183036 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.183011    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2339fb-serving-cert\""
Sep 14 05:19:16.187776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.187746    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:16.212011 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.211975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:16.228241 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.228200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.228241 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.228245    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s2rs\" (UniqueName: \"kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.228478 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.228300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.228478 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.228332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.269403 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.269365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:16.329483 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.329446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.329683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.329492    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s2rs\" (UniqueName: \"kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.329683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.329569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.329683 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.329616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.329866 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:19:16.329764    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-serving-cert: secret "maas-api-e2e-ipp-cleanup-2339fb-serving-cert" not found
Sep 14 05:19:16.329866 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:19:16.329848    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls podName:472912a0-52ef-444f-b094-0d13bcca338b nodeName:}" failed. No retries permitted until 2026-09-14 05:19:16.829823373 +0000 UTC m=+1951.191775806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" (UID: "472912a0-52ef-444f-b094-0d13bcca338b") : secret "maas-api-e2e-ipp-cleanup-2339fb-serving-cert" not found
Sep 14 05:19:16.330320 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.330301    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.332340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.332308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.338791 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.338762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s2rs\" (UniqueName: \"kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.527345 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.527307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:16.748750 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.748716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:16.835238 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.835133    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.837751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.837716    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:16.848912 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:16.848880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:17.096728 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.096624    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:17.185274 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.185241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:17.373590 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.373561    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:17.376754 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:19:17.376727    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472912a0_52ef_444f_b094_0d13bcca338b.slice/crio-167fb7987f431d1a91477b27628685b0258d942fd3407a8eda246f14349f4b9b WatchSource:0}: Error finding container 167fb7987f431d1a91477b27628685b0258d942fd3407a8eda246f14349f4b9b: Status 404 returned error can't find the container with id 167fb7987f431d1a91477b27628685b0258d942fd3407a8eda246f14349f4b9b
Sep 14 05:19:17.378781 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.378755    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 05:19:17.579004 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.578961    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" event={"ID":"472912a0-52ef-444f-b094-0d13bcca338b","Type":"ContainerStarted","Data":"167fb7987f431d1a91477b27628685b0258d942fd3407a8eda246f14349f4b9b"}
Sep 14 05:19:17.888480 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:17.888446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:18.587396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:18.587360    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" event={"ID":"472912a0-52ef-444f-b094-0d13bcca338b","Type":"ContainerStarted","Data":"a769bf550f960ea811f515cdd8bfb78d229328ffff15b535f6493480f9151946"}
Sep 14 05:19:18.587820 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:18.587427    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:18.609144 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:18.609080    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" podStartSLOduration=2.182637521 podStartE2EDuration="2.609061587s" totalImagesPullingTime="426.424066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:19:18.606533392 +0000 UTC m=+1952.968485832" watchObservedRunningTime="2026-09-14 05:19:18.609061587 +0000 UTC m=+1952.971014030"
Sep 14 05:19:24.607633 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:24.607598    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:38.782285 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:38.782187    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:39.912891 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:39.912857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:40.754112 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:40.754079    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:41.690410 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:41.690379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:42.574120 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:42.574088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:43.012847 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:43.012815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:43.349588 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:43.349506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:44.445774 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:44.445735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:45.239978 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:45.239920    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:46.002201 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:46.002149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:46.902670 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:46.902633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:47.663428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:47.663396    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:48.557458 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:48.557415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.089024 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.088988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.630441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.630411    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:54.630969 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.630916    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" podUID="472912a0-52ef-444f-b094-0d13bcca338b" containerName="maas-api" containerID="cri-o://a769bf550f960ea811f515cdd8bfb78d229328ffff15b535f6493480f9151946" gracePeriod=30
Sep 14 05:19:54.650464 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.650430    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.730295 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.730253    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.794184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.794127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.812754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.812615    2571 generic.go:356] "Generic (PLEG): container finished" podID="472912a0-52ef-444f-b094-0d13bcca338b" containerID="a769bf550f960ea811f515cdd8bfb78d229328ffff15b535f6493480f9151946" exitCode=0
Sep 14 05:19:54.812754 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.812656    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" event={"ID":"472912a0-52ef-444f-b094-0d13bcca338b","Type":"ContainerDied","Data":"a769bf550f960ea811f515cdd8bfb78d229328ffff15b535f6493480f9151946"}
Sep 14 05:19:54.871631 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.871593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:54.891096 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.891013    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:54.949761 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:54.949727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:55.001289 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001246    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca\") pod \"472912a0-52ef-444f-b094-0d13bcca338b\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") "
Sep 14 05:19:55.001499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001353    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs\") pod \"472912a0-52ef-444f-b094-0d13bcca338b\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") "
Sep 14 05:19:55.001499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001423    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s2rs\" (UniqueName: \"kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs\") pod \"472912a0-52ef-444f-b094-0d13bcca338b\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") "
Sep 14 05:19:55.001499 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001449    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") pod \"472912a0-52ef-444f-b094-0d13bcca338b\" (UID: \"472912a0-52ef-444f-b094-0d13bcca338b\") "
Sep 14 05:19:55.001661 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001633    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "472912a0-52ef-444f-b094-0d13bcca338b" (UID: "472912a0-52ef-444f-b094-0d13bcca338b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:19:55.001847 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.001826    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/472912a0-52ef-444f-b094-0d13bcca338b-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:19:55.003822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.003791    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "472912a0-52ef-444f-b094-0d13bcca338b" (UID: "472912a0-52ef-444f-b094-0d13bcca338b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:19:55.003977 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.003794    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs" (OuterVolumeSpecName: "kube-api-access-4s2rs") pod "472912a0-52ef-444f-b094-0d13bcca338b" (UID: "472912a0-52ef-444f-b094-0d13bcca338b"). InnerVolumeSpecName "kube-api-access-4s2rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:19:55.004126 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.004101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "472912a0-52ef-444f-b094-0d13bcca338b" (UID: "472912a0-52ef-444f-b094-0d13bcca338b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:19:55.103425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.103377    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s2rs\" (UniqueName: \"kubernetes.io/projected/472912a0-52ef-444f-b094-0d13bcca338b-kube-api-access-4s2rs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:19:55.103425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.103417    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:19:55.103425 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.103431    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/472912a0-52ef-444f-b094-0d13bcca338b-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:19:55.280018 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.279977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:55.383215 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.383180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:55.819695 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.819664    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"
Sep 14 05:19:55.819909 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.819700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4" event={"ID":"472912a0-52ef-444f-b094-0d13bcca338b","Type":"ContainerDied","Data":"167fb7987f431d1a91477b27628685b0258d942fd3407a8eda246f14349f4b9b"}
Sep 14 05:19:55.819909 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.819772    2571 scope.go:117] "RemoveContainer" containerID="a769bf550f960ea811f515cdd8bfb78d229328ffff15b535f6493480f9151946"
Sep 14 05:19:55.842842 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.842806    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:55.849399 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:55.849371    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2339fb-65df676556-5njl4"]
Sep 14 05:19:56.125579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:56.125484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:56.195491 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:56.195458    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="472912a0-52ef-444f-b094-0d13bcca338b" path="/var/lib/kubelet/pods/472912a0-52ef-444f-b094-0d13bcca338b/volumes"
Sep 14 05:19:56.862720 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:56.862680    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:57.744673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:57.744618    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:19:58.429547 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:19:58.429489    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:00.661711 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:00.661676    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:00.762015 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:00.761958    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:01.150843 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:01.150810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:01.212529 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:01.212498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:01.413398 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:01.413300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:05.645827 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:05.645786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:06.461124 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:06.461081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:07.173014 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:07.172900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:07.982191 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:07.982147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:08.783302 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:08.783264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:09.603268 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:09.603233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.020147 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.020105    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:20:21.020746 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.020381    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" podUID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" containerName="maas-api" containerID="cri-o://fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015" gracePeriod=30
Sep 14 05:20:21.047966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.047908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.121874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.121826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.180901 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.180862    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.276007 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.275909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.291904 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.291877    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:20:21.342243 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.342192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.461014 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.460968    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") pod \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") "
Sep 14 05:20:21.461216 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.461022    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqnmr\" (UniqueName: \"kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr\") pod \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") "
Sep 14 05:20:21.461216 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.461168    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs\") pod \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") "
Sep 14 05:20:21.461216 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.461198    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca\") pod \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\" (UID: \"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0\") "
Sep 14 05:20:21.461632 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.461601    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:20:21.463362 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.463337    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:20:21.463477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.463391    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:20:21.463477 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.463430    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr" (OuterVolumeSpecName: "kube-api-access-xqnmr") pod "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" (UID: "c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0"). InnerVolumeSpecName "kube-api-access-xqnmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:20:21.562855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.562765    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:20:21.562855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.562795    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnmr\" (UniqueName: \"kubernetes.io/projected/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-kube-api-access-xqnmr\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:20:21.562855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.562806    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:20:21.562855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.562817    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:20:21.660733 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.660698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.773429 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.773388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:21.968574 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.968530    2571 generic.go:356] "Generic (PLEG): container finished" podID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" containerID="fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015" exitCode=0
Sep 14 05:20:21.968751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.968583    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"
Sep 14 05:20:21.968751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.968599    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" event={"ID":"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0","Type":"ContainerDied","Data":"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"}
Sep 14 05:20:21.968751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.968654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf" event={"ID":"c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0","Type":"ContainerDied","Data":"8b511d41af5bd250e4a084485fcc7145ab63bd52192d21670ae54e79307e9adb"}
Sep 14 05:20:21.968751 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.968685    2571 scope.go:117] "RemoveContainer" containerID="fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"
Sep 14 05:20:21.979772 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.979751    2571 scope.go:117] "RemoveContainer" containerID="fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"
Sep 14 05:20:21.980180 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:20:21.980158    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015\": container with ID starting with fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015 not found: ID does not exist" containerID="fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"
Sep 14 05:20:21.980279 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.980191    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015"} err="failed to get container status \"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015\": rpc error: code = NotFound desc = could not find container \"fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015\": container with ID starting with fe80af11773bf1c387b13be70fe40708ee9ac8a42853e21389addacccebb8015 not found: ID does not exist"
Sep 14 05:20:21.992703 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.992665    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:20:21.999673 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:21.999643    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-524a38-c6644467-x28kf"]
Sep 14 05:20:22.195700 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:22.195660    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" path="/var/lib/kubelet/pods/c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0/volumes"
Sep 14 05:20:22.567719 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:22.567687    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:23.258338 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:23.258306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:24.081590 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:24.081528    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:26.684462 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:26.684428    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:39.863181 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:39.863147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:39.923811 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:39.923773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:51.428018 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:51.427984    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:20:51.533687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:20:51.533635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:03.232412 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:03.232370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:04.487193 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:04.487158    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:10.561699 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:10.561614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:10.668568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:10.668532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:10.953701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:10.953666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:11.557312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:11.557268    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:12.674219 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:12.674183    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:13.864608 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:13.864573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:14.663489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:14.663457    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:15.563604 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:15.563562    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:20.171601 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:20.171565    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:21.505590 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:21.505559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.101568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.101531    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:21:22.101829 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.101798    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerName="maas-api" containerID="cri-o://bb11563d1b844eeadcae23f8e93ca2b1bb140f01383413fffeadfcad4663b716" gracePeriod=30
Sep 14 05:21:22.128235 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.128199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.162293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.162245    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.81:8443/health\": dial tcp 10.134.0.81:8443: connect: connection refused"
Sep 14 05:21:22.193981 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.193900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.294832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.294801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.305280 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.305247    2571 generic.go:356] "Generic (PLEG): container finished" podID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerID="bb11563d1b844eeadcae23f8e93ca2b1bb140f01383413fffeadfcad4663b716" exitCode=0
Sep 14 05:21:22.305465 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.305337    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" event={"ID":"92c1d955-a942-4f5a-ba38-0792a37411ad","Type":"ContainerDied","Data":"bb11563d1b844eeadcae23f8e93ca2b1bb140f01383413fffeadfcad4663b716"}
Sep 14 05:21:22.370690 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.370665    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:21:22.395563 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.395527    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.445297 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.445257    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs\") pod \"92c1d955-a942-4f5a-ba38-0792a37411ad\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") "
Sep 14 05:21:22.445482 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.445382    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d525v\" (UniqueName: \"kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v\") pod \"92c1d955-a942-4f5a-ba38-0792a37411ad\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") "
Sep 14 05:21:22.445482 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.445402    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca\") pod \"92c1d955-a942-4f5a-ba38-0792a37411ad\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") "
Sep 14 05:21:22.445482 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.445436    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls\") pod \"92c1d955-a942-4f5a-ba38-0792a37411ad\" (UID: \"92c1d955-a942-4f5a-ba38-0792a37411ad\") "
Sep 14 05:21:22.445791 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.445760    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92c1d955-a942-4f5a-ba38-0792a37411ad" (UID: "92c1d955-a942-4f5a-ba38-0792a37411ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:21:22.448029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.447976    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "92c1d955-a942-4f5a-ba38-0792a37411ad" (UID: "92c1d955-a942-4f5a-ba38-0792a37411ad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:22.448332 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.448311    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92c1d955-a942-4f5a-ba38-0792a37411ad" (UID: "92c1d955-a942-4f5a-ba38-0792a37411ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:22.448429 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.448340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v" (OuterVolumeSpecName: "kube-api-access-d525v") pod "92c1d955-a942-4f5a-ba38-0792a37411ad" (UID: "92c1d955-a942-4f5a-ba38-0792a37411ad"). InnerVolumeSpecName "kube-api-access-d525v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:21:22.546393 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.546354    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d525v\" (UniqueName: \"kubernetes.io/projected/92c1d955-a942-4f5a-ba38-0792a37411ad-kube-api-access-d525v\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:22.546393 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.546387    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92c1d955-a942-4f5a-ba38-0792a37411ad-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:22.546393 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.546398    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:22.546851 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.546407    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92c1d955-a942-4f5a-ba38-0792a37411ad-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:22.759252 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.759222    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:22.855874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:22.855828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:23.311768 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.311728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d" event={"ID":"92c1d955-a942-4f5a-ba38-0792a37411ad","Type":"ContainerDied","Data":"c1b619ed4398db072ce5c773ff235a6094afe0f1813d2709c07d5e80efd91180"}
Sep 14 05:21:23.311768 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.311756    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"
Sep 14 05:21:23.312064 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.311784    2571 scope.go:117] "RemoveContainer" containerID="bb11563d1b844eeadcae23f8e93ca2b1bb140f01383413fffeadfcad4663b716"
Sep 14 05:21:23.334645 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.334605    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:21:23.340758 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.340727    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-7bbea9-6dcc87d67c-kq55d"]
Sep 14 05:21:23.770696 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:23.770657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:24.200966 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:24.196209    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" path="/var/lib/kubelet/pods/92c1d955-a942-4f5a-ba38-0792a37411ad/volumes"
Sep 14 05:21:24.772859 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:24.772820    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:28.838044 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:28.838006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:28.938703 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:28.938668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:29.151766 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:29.151662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:30.564045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:30.563974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:31.362454 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:31.362418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:32.063693 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:32.063658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.031566 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.031500    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:21:40.032441 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.032380    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" podUID="4b882d5f-a5c3-4781-b492-b7a117da1afb" containerName="maas-api" containerID="cri-o://e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e" gracePeriod=30
Sep 14 05:21:40.046229 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.046185    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.127687 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.127654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.191665 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.191623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.293237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.293204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.304438 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.304410    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:21:40.356011 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.355974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.407839 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.407795    2571 generic.go:356] "Generic (PLEG): container finished" podID="4b882d5f-a5c3-4781-b492-b7a117da1afb" containerID="e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e" exitCode=0
Sep 14 05:21:40.408073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.407878    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"
Sep 14 05:21:40.408073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.407892    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" event={"ID":"4b882d5f-a5c3-4781-b492-b7a117da1afb","Type":"ContainerDied","Data":"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"}
Sep 14 05:21:40.408073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.407970    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf" event={"ID":"4b882d5f-a5c3-4781-b492-b7a117da1afb","Type":"ContainerDied","Data":"b70fb6de6432b733dee0e8f7ee38c5578a79754d26d454e03772d0ee67c7d1c3"}
Sep 14 05:21:40.408073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.408005    2571 scope.go:117] "RemoveContainer" containerID="e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"
Sep 14 05:21:40.418567 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.418546    2571 scope.go:117] "RemoveContainer" containerID="e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"
Sep 14 05:21:40.418985 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:21:40.418960    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e\": container with ID starting with e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e not found: ID does not exist" containerID="e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"
Sep 14 05:21:40.419126 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.418991    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e"} err="failed to get container status \"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e\": rpc error: code = NotFound desc = could not find container \"e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e\": container with ID starting with e11aac70ebadfa798318959df32c99d232d2bfcf28d82e8226d2a6d56d93683e not found: ID does not exist"
Sep 14 05:21:40.429371 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.429337    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mssk\" (UniqueName: \"kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk\") pod \"4b882d5f-a5c3-4781-b492-b7a117da1afb\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") "
Sep 14 05:21:40.429535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.429428    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs\") pod \"4b882d5f-a5c3-4781-b492-b7a117da1afb\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") "
Sep 14 05:21:40.429535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.429465    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca\") pod \"4b882d5f-a5c3-4781-b492-b7a117da1afb\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") "
Sep 14 05:21:40.429535 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.429490    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") pod \"4b882d5f-a5c3-4781-b492-b7a117da1afb\" (UID: \"4b882d5f-a5c3-4781-b492-b7a117da1afb\") "
Sep 14 05:21:40.429852 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.429828    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b882d5f-a5c3-4781-b492-b7a117da1afb" (UID: "4b882d5f-a5c3-4781-b492-b7a117da1afb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:21:40.431799 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.431772    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b882d5f-a5c3-4781-b492-b7a117da1afb" (UID: "4b882d5f-a5c3-4781-b492-b7a117da1afb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:40.431889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.431830    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4b882d5f-a5c3-4781-b492-b7a117da1afb" (UID: "4b882d5f-a5c3-4781-b492-b7a117da1afb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:40.431889 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.431847    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk" (OuterVolumeSpecName: "kube-api-access-7mssk") pod "4b882d5f-a5c3-4781-b492-b7a117da1afb" (UID: "4b882d5f-a5c3-4781-b492-b7a117da1afb"). InnerVolumeSpecName "kube-api-access-7mssk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:21:40.531045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.531008    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:40.531045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.531036    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b882d5f-a5c3-4781-b492-b7a117da1afb-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:40.531045 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.531047    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b882d5f-a5c3-4781-b492-b7a117da1afb-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:40.531358 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.531079    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mssk\" (UniqueName: \"kubernetes.io/projected/4b882d5f-a5c3-4781-b492-b7a117da1afb-kube-api-access-7mssk\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:40.711440 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.711399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:40.730582 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.730548    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:21:40.734380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.734346    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-2ff4a6-7586874644-hbkcf"]
Sep 14 05:21:40.812408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:40.812366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:41.575377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:41.575345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:42.195378 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:42.195340    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b882d5f-a5c3-4781-b492-b7a117da1afb" path="/var/lib/kubelet/pods/4b882d5f-a5c3-4781-b492-b7a117da1afb/volumes"
Sep 14 05:21:46.296739 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.296705    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:21:46.301186 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.301160    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:21:46.302537 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.302510    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:21:46.306822 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.306803    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:21:46.913902 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.913863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:46.971374 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:46.971331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:47.194755 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:47.194668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:47.772876 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:47.772834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:48.651961 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:48.651897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:49.451733 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:49.451693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.080997 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.080913    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:21:58.081917 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.081781    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" podUID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" containerName="maas-api" containerID="cri-o://72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003" gracePeriod=30
Sep 14 05:21:58.111077 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.111035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.173187 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.173152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.289798 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.289764    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.345131 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.345060    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:21:58.346237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.346208    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.397676 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.397624    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bprzj\" (UniqueName: \"kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj\") pod \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") "
Sep 14 05:21:58.397893 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.397690    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca\") pod \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") "
Sep 14 05:21:58.397893 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.397781    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") pod \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") "
Sep 14 05:21:58.397893 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.397853    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs\") pod \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\" (UID: \"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47\") "
Sep 14 05:21:58.398184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.398142    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" (UID: "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:21:58.400428 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.400396    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" (UID: "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:58.400666 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.400634    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj" (OuterVolumeSpecName: "kube-api-access-bprzj") pod "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" (UID: "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47"). InnerVolumeSpecName "kube-api-access-bprzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:21:58.400666 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.400631    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" (UID: "e0de8bc9-8a60-41c9-8a3a-f1e028a23e47"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:21:58.499372 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.499333    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:58.499372 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.499367    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:58.499372 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.499379    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bprzj\" (UniqueName: \"kubernetes.io/projected/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-kube-api-access-bprzj\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:58.499627 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.499389    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:21:58.506123 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.506089    2571 generic.go:356] "Generic (PLEG): container finished" podID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" containerID="72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003" exitCode=0
Sep 14 05:21:58.506342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.506130    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"
Sep 14 05:21:58.506342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.506185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" event={"ID":"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47","Type":"ContainerDied","Data":"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"}
Sep 14 05:21:58.506342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.506249    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7" event={"ID":"e0de8bc9-8a60-41c9-8a3a-f1e028a23e47","Type":"ContainerDied","Data":"6eeeb7389b16e896de4ea1e052dbe66cfe874017115afa651ab4c8f0b82f3923"}
Sep 14 05:21:58.506342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.506284    2571 scope.go:117] "RemoveContainer" containerID="72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"
Sep 14 05:21:58.516315 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.516294    2571 scope.go:117] "RemoveContainer" containerID="72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"
Sep 14 05:21:58.516625 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:21:58.516607    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003\": container with ID starting with 72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003 not found: ID does not exist" containerID="72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"
Sep 14 05:21:58.516681 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.516635    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003"} err="failed to get container status \"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003\": rpc error: code = NotFound desc = could not find container \"72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003\": container with ID starting with 72ed4eb5b9d3c5a71e39324e5aaa6dd171389112bf4bc317aa10f19acd8dc003 not found: ID does not exist"
Sep 14 05:21:58.528293 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.528254    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:21:58.531898 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.531864    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-27be50-74cfd44889-6srm7"]
Sep 14 05:21:58.744300 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.744267    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:21:58.845556 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:21:58.845520    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:22:00.195117 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:00.195075    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" path="/var/lib/kubelet/pods/e0de8bc9-8a60-41c9-8a3a-f1e028a23e47/volumes"
Sep 14 05:22:04.840730 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:04.840694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:22:04.941458 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:04.941422    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:22:05.128261 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:05.128181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:22:09.315112 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.315066    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 05:22:09.315654 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.315351    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c44665c5c-757ct" podUID="f44552da-2452-4dc5-a431-2ae33baef91d" containerName="manager" containerID="cri-o://588764346ff236edd16b4a2f8cb9640082fc6a01a2a43051eb79162a02ad1688" gracePeriod=30
Sep 14 05:22:09.566270 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.566181    2571 generic.go:356] "Generic (PLEG): container finished" podID="f44552da-2452-4dc5-a431-2ae33baef91d" containerID="588764346ff236edd16b4a2f8cb9640082fc6a01a2a43051eb79162a02ad1688" exitCode=0
Sep 14 05:22:09.566270 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.566240    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-757ct" event={"ID":"f44552da-2452-4dc5-a431-2ae33baef91d","Type":"ContainerDied","Data":"588764346ff236edd16b4a2f8cb9640082fc6a01a2a43051eb79162a02ad1688"}
Sep 14 05:22:09.566486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.566280    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-757ct" event={"ID":"f44552da-2452-4dc5-a431-2ae33baef91d","Type":"ContainerDied","Data":"a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5"}
Sep 14 05:22:09.566486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.566296    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a690e8747ddccd73b6c793d32d40e6c8b3903a2bdd158004f0db9a9c479ea0c5"
Sep 14 05:22:09.575314 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.575285    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 05:22:09.702340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.702296    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert\") pod \"f44552da-2452-4dc5-a431-2ae33baef91d\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") "
Sep 14 05:22:09.702340 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.702343    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm6f8\" (UniqueName: \"kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8\") pod \"f44552da-2452-4dc5-a431-2ae33baef91d\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") "
Sep 14 05:22:09.702602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.702369    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs\") pod \"f44552da-2452-4dc5-a431-2ae33baef91d\" (UID: \"f44552da-2452-4dc5-a431-2ae33baef91d\") "
Sep 14 05:22:09.704770 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.704729    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f44552da-2452-4dc5-a431-2ae33baef91d" (UID: "f44552da-2452-4dc5-a431-2ae33baef91d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:22:09.704887 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.704771    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f44552da-2452-4dc5-a431-2ae33baef91d" (UID: "f44552da-2452-4dc5-a431-2ae33baef91d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:22:09.704887 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.704778    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8" (OuterVolumeSpecName: "kube-api-access-nm6f8") pod "f44552da-2452-4dc5-a431-2ae33baef91d" (UID: "f44552da-2452-4dc5-a431-2ae33baef91d"). InnerVolumeSpecName "kube-api-access-nm6f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:22:09.803662 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.803619    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-webhook-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:22:09.803662 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.803654    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm6f8\" (UniqueName: \"kubernetes.io/projected/f44552da-2452-4dc5-a431-2ae33baef91d-kube-api-access-nm6f8\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:22:09.803662 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:09.803663    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f44552da-2452-4dc5-a431-2ae33baef91d-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:22:10.571333 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:10.571228    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-757ct"
Sep 14 05:22:10.588229 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:10.588194    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 05:22:10.594532 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:10.594502    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-757ct"]
Sep 14 05:22:12.194611 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:12.194566    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f44552da-2452-4dc5-a431-2ae33baef91d" path="/var/lib/kubelet/pods/f44552da-2452-4dc5-a431-2ae33baef91d/volumes"
Sep 14 05:22:46.536392 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:22:46.536308    2571 scope.go:117] "RemoveContainer" containerID="588764346ff236edd16b4a2f8cb9640082fc6a01a2a43051eb79162a02ad1688"
Sep 14 05:23:05.652245 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:05.652209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:23:11.436967 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.436879    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:11.437675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437628    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" containerName="maas-api"
Sep 14 05:23:11.437675 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437665    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437693    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b882d5f-a5c3-4781-b492-b7a117da1afb" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437700    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b882d5f-a5c3-4781-b492-b7a117da1afb" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437707    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437720    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437726    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="472912a0-52ef-444f-b094-0d13bcca338b" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437731    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="472912a0-52ef-444f-b094-0d13bcca338b" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437741    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f44552da-2452-4dc5-a431-2ae33baef91d" containerName="manager"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437751    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f44552da-2452-4dc5-a431-2ae33baef91d" containerName="manager"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" containerName="maas-api"
Sep 14 05:23:11.437832 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437772    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" containerName="maas-api"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437908    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5db76b2-8dcf-4f8b-a1d0-dd4986c82fc0" containerName="maas-api"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437925    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b882d5f-a5c3-4781-b492-b7a117da1afb" containerName="maas-api"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437973    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f44552da-2452-4dc5-a431-2ae33baef91d" containerName="manager"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437982    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="472912a0-52ef-444f-b094-0d13bcca338b" containerName="maas-api"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.437993    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0de8bc9-8a60-41c9-8a3a-f1e028a23e47" containerName="maas-api"
Sep 14 05:23:11.438262 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.438003    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c1d955-a942-4f5a-ba38-0792a37411ad" containerName="maas-api"
Sep 14 05:23:11.441396 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.441379    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.445325 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.445300    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 05:23:11.445466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.445333    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 05:23:11.445466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.445338    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 05:23:11.445466 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.445308    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7hwt9\""
Sep 14 05:23:11.450862 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.450838    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:11.571012 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.570920    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.571202 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.571043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.571202 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.571073    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bff8b\" (UniqueName: \"kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.671918 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.671862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.671918 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.671954    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.672180 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.671978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bff8b\" (UniqueName: \"kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.674580 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.674548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.674697 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.674554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.684992 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.684962    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bff8b\" (UniqueName: \"kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b\") pod \"maas-controller-6c44665c5c-t5zt9\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") " pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:11.753799 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:11.753755    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:12.029834 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:12.029800    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:12.032629 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:23:12.032597    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0182014c_170c_4505_92f1_9c468381998f.slice/crio-4bd2e86a1e18406ebeecaade6ac639b7e23bf9b9f1bac02cef503299120af605 WatchSource:0}: Error finding container 4bd2e86a1e18406ebeecaade6ac639b7e23bf9b9f1bac02cef503299120af605: Status 404 returned error can't find the container with id 4bd2e86a1e18406ebeecaade6ac639b7e23bf9b9f1bac02cef503299120af605
Sep 14 05:23:12.910575 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:12.910469    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" event={"ID":"0182014c-170c-4505-92f1-9c468381998f","Type":"ContainerStarted","Data":"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"}
Sep 14 05:23:12.910575 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:12.910526    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" event={"ID":"0182014c-170c-4505-92f1-9c468381998f","Type":"ContainerStarted","Data":"4bd2e86a1e18406ebeecaade6ac639b7e23bf9b9f1bac02cef503299120af605"}
Sep 14 05:23:12.910575 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:12.910549    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:12.929978 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:12.929894    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" podStartSLOduration=1.444613323 podStartE2EDuration="1.929874905s" totalImagesPullingTime="485.261582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:23:12.927991813 +0000 UTC m=+2187.289944259" watchObservedRunningTime="2026-09-14 05:23:12.929874905 +0000 UTC m=+2187.291827369"
Sep 14 05:23:23.918776 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:23.918745    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:33.136578 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.136532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:23:33.229814 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.229775    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:33.230155 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.230117    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" podUID="0182014c-170c-4505-92f1-9c468381998f" containerName="manager" containerID="cri-o://305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1" gracePeriod=30
Sep 14 05:23:33.492307 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.492273    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:33.572580 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.572545    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bff8b\" (UniqueName: \"kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b\") pod \"0182014c-170c-4505-92f1-9c468381998f\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") "
Sep 14 05:23:33.572798 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.572639    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert\") pod \"0182014c-170c-4505-92f1-9c468381998f\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") "
Sep 14 05:23:33.572798 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.572723    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs\") pod \"0182014c-170c-4505-92f1-9c468381998f\" (UID: \"0182014c-170c-4505-92f1-9c468381998f\") "
Sep 14 05:23:33.575515 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.575465    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0182014c-170c-4505-92f1-9c468381998f" (UID: "0182014c-170c-4505-92f1-9c468381998f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:23:33.575707 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.575560    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b" (OuterVolumeSpecName: "kube-api-access-bff8b") pod "0182014c-170c-4505-92f1-9c468381998f" (UID: "0182014c-170c-4505-92f1-9c468381998f"). InnerVolumeSpecName "kube-api-access-bff8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:23:33.575707 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.575582    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0182014c-170c-4505-92f1-9c468381998f" (UID: "0182014c-170c-4505-92f1-9c468381998f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:23:33.673946 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.673825    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:23:33.673946 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.673861    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bff8b\" (UniqueName: \"kubernetes.io/projected/0182014c-170c-4505-92f1-9c468381998f-kube-api-access-bff8b\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:23:33.673946 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:33.673871    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0182014c-170c-4505-92f1-9c468381998f-webhook-cert\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:23:34.026312 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.026276    2571 generic.go:356] "Generic (PLEG): container finished" podID="0182014c-170c-4505-92f1-9c468381998f" containerID="305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1" exitCode=0
Sep 14 05:23:34.026500 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.026335    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-t5zt9"
Sep 14 05:23:34.026500 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.026347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" event={"ID":"0182014c-170c-4505-92f1-9c468381998f","Type":"ContainerDied","Data":"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"}
Sep 14 05:23:34.026500 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.026393    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-t5zt9" event={"ID":"0182014c-170c-4505-92f1-9c468381998f","Type":"ContainerDied","Data":"4bd2e86a1e18406ebeecaade6ac639b7e23bf9b9f1bac02cef503299120af605"}
Sep 14 05:23:34.026500 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.026416    2571 scope.go:117] "RemoveContainer" containerID="305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"
Sep 14 05:23:34.036073 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.036054    2571 scope.go:117] "RemoveContainer" containerID="305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"
Sep 14 05:23:34.036374 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:23:34.036354    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1\": container with ID starting with 305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1 not found: ID does not exist" containerID="305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"
Sep 14 05:23:34.036433 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.036384    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1"} err="failed to get container status \"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1\": rpc error: code = NotFound desc = could not find container \"305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1\": container with ID starting with 305e367d4bf0d89cf9b2e9d6f0e2d271efbae59cd0572fa6b87932e7a085f4e1 not found: ID does not exist"
Sep 14 05:23:34.049659 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.049627    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:34.054106 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.054079    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-t5zt9"]
Sep 14 05:23:34.194645 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.194612    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0182014c-170c-4505-92f1-9c468381998f" path="/var/lib/kubelet/pods/0182014c-170c-4505-92f1-9c468381998f/volumes"
Sep 14 05:23:34.718038 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.718001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:23:34.831796 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.831762    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c44665c5c-5s4tn"]
Sep 14 05:23:34.832188 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.832174    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0182014c-170c-4505-92f1-9c468381998f" containerName="manager"
Sep 14 05:23:34.832249 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.832189    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0182014c-170c-4505-92f1-9c468381998f" containerName="manager"
Sep 14 05:23:34.832283 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.832255    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0182014c-170c-4505-92f1-9c468381998f" containerName="manager"
Sep 14 05:23:34.836712 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.836690    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.839667 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.839634    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7hwt9\""
Sep 14 05:23:34.839667 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.839651    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 05:23:34.840986 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.840968    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 05:23:34.840986 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.840981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 05:23:34.846377 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.846351    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-5s4tn"]
Sep 14 05:23:34.885701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.885650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntrs2\" (UniqueName: \"kubernetes.io/projected/0cb003ca-953e-47b1-85ec-7479a880812a-kube-api-access-ntrs2\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.885701 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.885697    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-webhook-cert\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.886001 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.885788    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-metrics-certs\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.987388 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.987288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntrs2\" (UniqueName: \"kubernetes.io/projected/0cb003ca-953e-47b1-85ec-7479a880812a-kube-api-access-ntrs2\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.987388 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.987338    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-webhook-cert\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.987388 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.987382    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-metrics-certs\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.990209 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.990174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-webhook-cert\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.990335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.990174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0cb003ca-953e-47b1-85ec-7479a880812a-metrics-certs\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:34.997689 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:34.997663    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntrs2\" (UniqueName: \"kubernetes.io/projected/0cb003ca-953e-47b1-85ec-7479a880812a-kube-api-access-ntrs2\") pod \"maas-controller-6c44665c5c-5s4tn\" (UID: \"0cb003ca-953e-47b1-85ec-7479a880812a\") " pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:35.149504 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:35.149463    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:35.426002 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:35.425978    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c44665c5c-5s4tn"]
Sep 14 05:23:35.428132 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:23:35.428103    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb003ca_953e_47b1_85ec_7479a880812a.slice/crio-50da438cb00dfb94a743bd009b2ec655ba3e8fd959ca641fdd883802f3d7664f WatchSource:0}: Error finding container 50da438cb00dfb94a743bd009b2ec655ba3e8fd959ca641fdd883802f3d7664f: Status 404 returned error can't find the container with id 50da438cb00dfb94a743bd009b2ec655ba3e8fd959ca641fdd883802f3d7664f
Sep 14 05:23:36.046789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:36.046689    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-5s4tn" event={"ID":"0cb003ca-953e-47b1-85ec-7479a880812a","Type":"ContainerStarted","Data":"fc4fc0e6634093db64cdc7fd9ec915f51995b02e89b9ab69c051e03b25094b03"}
Sep 14 05:23:36.046789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:36.046778    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c44665c5c-5s4tn" event={"ID":"0cb003ca-953e-47b1-85ec-7479a880812a","Type":"ContainerStarted","Data":"50da438cb00dfb94a743bd009b2ec655ba3e8fd959ca641fdd883802f3d7664f"}
Sep 14 05:23:36.047081 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:36.046829    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:23:36.064948 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:36.064876    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c44665c5c-5s4tn" podStartSLOduration=1.767113741 podStartE2EDuration="2.064860421s" totalImagesPullingTime="297.74668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:23:36.064561605 +0000 UTC m=+2210.426514061" watchObservedRunningTime="2026-09-14 05:23:36.064860421 +0000 UTC m=+2210.426812863"
Sep 14 05:23:47.060607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:23:47.060570    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c44665c5c-5s4tn"
Sep 14 05:24:45.187248 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:45.187208    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 05:24:45.188183 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:45.188144    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" podUID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" containerName="manager" containerID="cri-o://1a5dd21851a2ae9d9a7e5803ca500b915b12718963df715e618f033594ebb215" gracePeriod=10
Sep 14 05:24:46.412436 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.412402    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" containerID="1a5dd21851a2ae9d9a7e5803ca500b915b12718963df715e618f033594ebb215" exitCode=0
Sep 14 05:24:46.412833 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.412483    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" event={"ID":"6a5de6f0-d879-46bc-9efc-391fefa2ded3","Type":"ContainerDied","Data":"1a5dd21851a2ae9d9a7e5803ca500b915b12718963df715e618f033594ebb215"}
Sep 14 05:24:46.847033 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.847000    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 05:24:46.970445 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.970339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume\") pod \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") "
Sep 14 05:24:46.970445 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.970425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtgj8\" (UniqueName: \"kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8\") pod \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\" (UID: \"6a5de6f0-d879-46bc-9efc-391fefa2ded3\") "
Sep 14 05:24:46.970723 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.970689    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a5de6f0-d879-46bc-9efc-391fefa2ded3" (UID: "6a5de6f0-d879-46bc-9efc-391fefa2ded3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 05:24:46.970799 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.970780    2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a5de6f0-d879-46bc-9efc-391fefa2ded3-extensions-socket-volume\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:24:46.972654 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:46.972630    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8" (OuterVolumeSpecName: "kube-api-access-mtgj8") pod "6a5de6f0-d879-46bc-9efc-391fefa2ded3" (UID: "6a5de6f0-d879-46bc-9efc-391fefa2ded3"). InnerVolumeSpecName "kube-api-access-mtgj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:24:47.071299 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.071258    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtgj8\" (UniqueName: \"kubernetes.io/projected/6a5de6f0-d879-46bc-9efc-391fefa2ded3-kube-api-access-mtgj8\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:24:47.419264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.419172    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"
Sep 14 05:24:47.419264 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.419208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp" event={"ID":"6a5de6f0-d879-46bc-9efc-391fefa2ded3","Type":"ContainerDied","Data":"385f5f1c46cee6eb8ca80e8ec0a4beb3345057f4efd3fcf1c9c768db82deae53"}
Sep 14 05:24:47.419768 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.419274    2571 scope.go:117] "RemoveContainer" containerID="1a5dd21851a2ae9d9a7e5803ca500b915b12718963df715e618f033594ebb215"
Sep 14 05:24:47.441213 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.441176    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 05:24:47.444817 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:47.444784    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-b92qp"]
Sep 14 05:24:48.194594 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:24:48.194549    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" path="/var/lib/kubelet/pods/6a5de6f0-d879-46bc-9efc-391fefa2ded3/volumes"
Sep 14 05:25:45.917351 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.917269    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"]
Sep 14 05:25:45.917785 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.917706    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" containerName="manager"
Sep 14 05:25:45.917785 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.917724    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" containerName="manager"
Sep 14 05:25:45.917864 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.917816    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a5de6f0-d879-46bc-9efc-391fefa2ded3" containerName="manager"
Sep 14 05:25:45.921006 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.920987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:45.924040 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.924024    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-29dd4\""
Sep 14 05:25:45.935514 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:45.935486    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"]
Sep 14 05:25:46.109813 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.109767    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zx6r\" (UniqueName: \"kubernetes.io/projected/74d5f909-7a5c-4de2-b921-391d6659cb7e-kube-api-access-7zx6r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.110000 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.109836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d5f909-7a5c-4de2-b921-391d6659cb7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.210961 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.210896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx6r\" (UniqueName: \"kubernetes.io/projected/74d5f909-7a5c-4de2-b921-391d6659cb7e-kube-api-access-7zx6r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.210961 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.210967    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d5f909-7a5c-4de2-b921-391d6659cb7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.211439 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.211407    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74d5f909-7a5c-4de2-b921-391d6659cb7e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.220391 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.220362    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zx6r\" (UniqueName: \"kubernetes.io/projected/74d5f909-7a5c-4de2-b921-391d6659cb7e-kube-api-access-7zx6r\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6qh8q\" (UID: \"74d5f909-7a5c-4de2-b921-391d6659cb7e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.234171 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.234141    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-29dd4\""
Sep 14 05:25:46.241349 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.241324    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.523603 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.523578    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"]
Sep 14 05:25:46.526207 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:25:46.526173    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d5f909_7a5c_4de2_b921_391d6659cb7e.slice/crio-af57dad225b6690a2bc5bbf2a5fc9e421da8b89fb246ad9160c3b02dad547af0 WatchSource:0}: Error finding container af57dad225b6690a2bc5bbf2a5fc9e421da8b89fb246ad9160c3b02dad547af0: Status 404 returned error can't find the container with id af57dad225b6690a2bc5bbf2a5fc9e421da8b89fb246ad9160c3b02dad547af0
Sep 14 05:25:46.722412 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.722372    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q" event={"ID":"74d5f909-7a5c-4de2-b921-391d6659cb7e","Type":"ContainerStarted","Data":"3865e68bfe45627a8845eeb1f87e60742604286fdd247412dc3234f266289a71"}
Sep 14 05:25:46.722607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.722424    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q" event={"ID":"74d5f909-7a5c-4de2-b921-391d6659cb7e","Type":"ContainerStarted","Data":"af57dad225b6690a2bc5bbf2a5fc9e421da8b89fb246ad9160c3b02dad547af0"}
Sep 14 05:25:46.722607 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.722487    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:25:46.743663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:46.743608    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q" podStartSLOduration=1.743582239 podStartE2EDuration="1.743582239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:25:46.741515077 +0000 UTC m=+2341.103467541" watchObservedRunningTime="2026-09-14 05:25:46.743582239 +0000 UTC m=+2341.105534684"
Sep 14 05:25:57.731040 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:25:57.731006    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6qh8q"
Sep 14 05:26:15.499779 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:15.499743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:24.041884 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:24.041841    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:33.848839 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:33.848805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:39.833591 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:39.833535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:46.330122 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:46.330086    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:26:46.335100 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:46.335076    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:26:46.335450 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:46.335430    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:26:46.340486 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:46.340465    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n7vbm_3d32c872-6925-475a-8500-7029bc315861/ovn-acl-logging/0.log"
Sep 14 05:26:50.850635 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:50.850603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:50.915012 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:50.914979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:26:53.647830 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:26:53.647790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:27:05.839324 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:05.839284    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:27:05.895759 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:05.895720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:27:21.877268 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:21.877231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:27:22.625564 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:22.625532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:27:42.739342 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.739308    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"]
Sep 14 05:27:42.743156 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.743129    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.752447 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.752415    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"]
Sep 14 05:27:42.797335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.797291    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-metrics-certs\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.797335 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.797340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-maas-api-tls\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.797576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.797413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d9e1de-d98f-46de-9682-55a691294d1e-openshift-service-ca\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.797576 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.797449    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kctvj\" (UniqueName: \"kubernetes.io/projected/f6d9e1de-d98f-46de-9682-55a691294d1e-kube-api-access-kctvj\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.899018 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.898970    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-metrics-certs\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.899246 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.899034    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-maas-api-tls\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.899246 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.899095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d9e1de-d98f-46de-9682-55a691294d1e-openshift-service-ca\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.899246 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.899139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kctvj\" (UniqueName: \"kubernetes.io/projected/f6d9e1de-d98f-46de-9682-55a691294d1e-kube-api-access-kctvj\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.900012 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.899986    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6d9e1de-d98f-46de-9682-55a691294d1e-openshift-service-ca\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.902280 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.902248    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-metrics-certs\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.902506 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.902483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6d9e1de-d98f-46de-9682-55a691294d1e-maas-api-tls\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:42.907874 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:42.907843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kctvj\" (UniqueName: \"kubernetes.io/projected/f6d9e1de-d98f-46de-9682-55a691294d1e-kube-api-access-kctvj\") pod \"maas-api-66f5596c4d-qz7dw\" (UID: \"f6d9e1de-d98f-46de-9682-55a691294d1e\") " pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:43.055359 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:43.055260    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:43.334087 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:43.334059    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"]
Sep 14 05:27:43.336674 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:27:43.336633    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d9e1de_d98f_46de_9682_55a691294d1e.slice/crio-9970f448c831079689c3c7077b38d1194871b84732a35f8eb7e403194608d776 WatchSource:0}: Error finding container 9970f448c831079689c3c7077b38d1194871b84732a35f8eb7e403194608d776: Status 404 returned error can't find the container with id 9970f448c831079689c3c7077b38d1194871b84732a35f8eb7e403194608d776
Sep 14 05:27:43.338243 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:43.338214    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 05:27:43.347235 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:43.347201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw" event={"ID":"f6d9e1de-d98f-46de-9682-55a691294d1e","Type":"ContainerStarted","Data":"9970f448c831079689c3c7077b38d1194871b84732a35f8eb7e403194608d776"}
Sep 14 05:27:44.354427 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:44.354387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw" event={"ID":"f6d9e1de-d98f-46de-9682-55a691294d1e","Type":"ContainerStarted","Data":"42cdf3805c31c28f4d78ea316f485c7a2c78f79ea3b8ac8c9abadead8cf566e9"}
Sep 14 05:27:44.354837 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:44.354521    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:44.372969 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:44.372890    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw" podStartSLOduration=1.9952293540000001 podStartE2EDuration="2.37287324s" totalImagesPullingTime="377.643886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 05:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:27:44.371707708 +0000 UTC m=+2458.733660152" watchObservedRunningTime="2026-09-14 05:27:44.37287324 +0000 UTC m=+2458.734825682"
Sep 14 05:27:50.364257 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:50.364227    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-66f5596c4d-qz7dw"
Sep 14 05:27:59.580173 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:59.580133    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 05:27:59.580652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:27:59.580398    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" podUID="454f9f2a-f35c-4658-bd10-5487c8751089" containerName="maas-api" containerID="cri-o://535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3" gracePeriod=30
Sep 14 05:28:00.833325 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.833297    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 05:28:00.969417 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.969379    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ghn8\" (UniqueName: \"kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8\") pod \"454f9f2a-f35c-4658-bd10-5487c8751089\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") "
Sep 14 05:28:00.969616 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.969446    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs\") pod \"454f9f2a-f35c-4658-bd10-5487c8751089\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") "
Sep 14 05:28:00.969616 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.969475    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca\") pod \"454f9f2a-f35c-4658-bd10-5487c8751089\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") "
Sep 14 05:28:00.969616 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.969576    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls\") pod \"454f9f2a-f35c-4658-bd10-5487c8751089\" (UID: \"454f9f2a-f35c-4658-bd10-5487c8751089\") "
Sep 14 05:28:00.969877 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.969853    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "454f9f2a-f35c-4658-bd10-5487c8751089" (UID: "454f9f2a-f35c-4658-bd10-5487c8751089"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 05:28:00.971958 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.971899    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "454f9f2a-f35c-4658-bd10-5487c8751089" (UID: "454f9f2a-f35c-4658-bd10-5487c8751089"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:28:00.971958 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.971913    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "454f9f2a-f35c-4658-bd10-5487c8751089" (UID: "454f9f2a-f35c-4658-bd10-5487c8751089"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 05:28:00.972135 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:00.971968    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8" (OuterVolumeSpecName: "kube-api-access-7ghn8") pod "454f9f2a-f35c-4658-bd10-5487c8751089" (UID: "454f9f2a-f35c-4658-bd10-5487c8751089"). InnerVolumeSpecName "kube-api-access-7ghn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 05:28:01.070855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.070811    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ghn8\" (UniqueName: \"kubernetes.io/projected/454f9f2a-f35c-4658-bd10-5487c8751089-kube-api-access-7ghn8\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:28:01.070855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.070848    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-metrics-certs\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:28:01.070855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.070857    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/454f9f2a-f35c-4658-bd10-5487c8751089-openshift-service-ca\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:28:01.070855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.070865    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/454f9f2a-f35c-4658-bd10-5487c8751089-maas-api-tls\") on node \"ip-10-0-137-105.ec2.internal\" DevicePath \"\""
Sep 14 05:28:01.448742 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.448648    2571 generic.go:356] "Generic (PLEG): container finished" podID="454f9f2a-f35c-4658-bd10-5487c8751089" containerID="535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3" exitCode=0
Sep 14 05:28:01.448742 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.448698    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf"
Sep 14 05:28:01.448985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.448736    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" event={"ID":"454f9f2a-f35c-4658-bd10-5487c8751089","Type":"ContainerDied","Data":"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"}
Sep 14 05:28:01.448985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.448788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f795d4696-826mf" event={"ID":"454f9f2a-f35c-4658-bd10-5487c8751089","Type":"ContainerDied","Data":"6b9cd12df8e98c10d883edfaaa359686f354ac873273281eb338533b3475b320"}
Sep 14 05:28:01.448985 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.448812    2571 scope.go:117] "RemoveContainer" containerID="535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"
Sep 14 05:28:01.458665 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.458646    2571 scope.go:117] "RemoveContainer" containerID="535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"
Sep 14 05:28:01.458996 ip-10-0-137-105 kubenswrapper[2571]: E0914 05:28:01.458972    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3\": container with ID starting with 535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3 not found: ID does not exist" containerID="535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"
Sep 14 05:28:01.459142 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.459010    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3"} err="failed to get container status \"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3\": rpc error: code = NotFound desc = could not find container \"535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3\": container with ID starting with 535925b2b686daaf54ae47d6c4a529b44d9c134b5b7b513babbe8e8c3fb35ae3 not found: ID does not exist"
Sep 14 05:28:01.469549 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.469520    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 05:28:01.474990 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:01.474964    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f795d4696-826mf"]
Sep 14 05:28:02.197150 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:02.197109    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="454f9f2a-f35c-4658-bd10-5487c8751089" path="/var/lib/kubelet/pods/454f9f2a-f35c-4658-bd10-5487c8751089/volumes"
Sep 14 05:28:18.300553 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.300518    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5jl2t_50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb/manager/0.log"
Sep 14 05:28:18.428380 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.428337    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ttvrr_c16eebb5-a63b-4538-bf06-045b63b7d9db/manager/0.log"
Sep 14 05:28:18.560421 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.560331    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c44665c5c-5s4tn_0cb003ca-953e-47b1-85ec-7479a880812a/manager/0.log"
Sep 14 05:28:18.681481 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.681450    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5wwgd_8ef3c739-010f-48a7-a2a4-318b70c37d4d/server/0.log"
Sep 14 05:28:18.940471 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.940391    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b6456dc5-p42j6_282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a/copy-manifests/0.log"
Sep 14 05:28:18.952500 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:18.952459    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b6456dc5-p42j6_282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a/manager/0.log"
Sep 14 05:28:19.672899 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:19.672865    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-66f5596c4d-qz7dw_f6d9e1de-d98f-46de-9682-55a691294d1e/maas-api/0.log"
Sep 14 05:28:20.031445 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:20.031413    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-rzcdw_7beb41a7-9ae2-4fca-9d3a-d69a247cb727/postgres/0.log"
Sep 14 05:28:20.986517 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:20.986481    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2hfcm_57b59a14-4a57-49de-bc36-9888741c0dcc/manager/0.log"
Sep 14 05:28:21.105819 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:21.105789    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-q2fl6_03f32217-adf1-4e92-82ce-baf9cb82c543/manager/0.log"
Sep 14 05:28:21.220734 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:21.220707    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-k26jx_06850d28-9a58-4833-9687-30f9f016efe5/kuadrant-console-plugin/0.log"
Sep 14 05:28:21.346828 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:21.346744    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6qh8q_74d5f909-7a5c-4de2-b921-391d6659cb7e/manager/0.log"
Sep 14 05:28:21.616517 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:21.616427    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-8nwc5_44961df4-cacd-4bac-acf8-ad26323b4c18/manager/0.log"
Sep 14 05:28:22.220449 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:22.220420    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96745c74f-dttph_82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04/kube-auth-proxy/0.log"
Sep 14 05:28:22.570345 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:22.570264    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gtt2g_cda02c30-fd51-4775-9881-11d9b3493245/payload-pre-processing/0.log"
Sep 14 05:28:23.139237 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:23.139208    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9q8rh_c8a4f8ab-394b-4294-b0c6-b9b8b864830a/main/0.log"
Sep 14 05:28:23.471789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:23.471760    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vqhh6_a85ecbf3-8081-4a0f-a87f-86e673ead417/main/0.log"
Sep 14 05:28:29.455782 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:29.455743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:28:29.513484 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:29.513452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log" limit=1024
Sep 14 05:28:32.439420 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:32.439383    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n7j5q_8f38c3e0-c0fb-4796-84c7-6c483ab48e2b/global-pull-secret-syncer/0.log"
Sep 14 05:28:32.533806 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:32.533777    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jpwnd_ef80e464-bdf3-47ea-b084-bd8f06a17739/konnectivity-agent/0.log"
Sep 14 05:28:32.679690 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:32.679653    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-105.ec2.internal_4cd3bb05a5d106b39c55a9ac721c0cb9/haproxy/0.log"
Sep 14 05:28:37.142016 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:37.141920    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2hfcm_57b59a14-4a57-49de-bc36-9888741c0dcc/manager/0.log"
Sep 14 05:28:37.170612 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:37.170586    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-q2fl6_03f32217-adf1-4e92-82ce-baf9cb82c543/manager/0.log"
Sep 14 05:28:37.194880 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:37.194848    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-k26jx_06850d28-9a58-4833-9687-30f9f016efe5/kuadrant-console-plugin/0.log"
Sep 14 05:28:37.256415 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:37.256379    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6qh8q_74d5f909-7a5c-4de2-b921-391d6659cb7e/manager/0.log"
Sep 14 05:28:37.368839 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:37.368800    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-8nwc5_44961df4-cacd-4bac-acf8-ad26323b4c18/manager/0.log"
Sep 14 05:28:39.206602 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.206575    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-f2h9t_83daf0c8-91ee-4345-8ef2-97628c929cfe/monitoring-plugin/0.log"
Sep 14 05:28:39.326671 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.326634    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cslbk_f1554549-75e9-4d27-b4c5-c8b2d0e58201/node-exporter/0.log"
Sep 14 05:28:39.367184 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.367139    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cslbk_f1554549-75e9-4d27-b4c5-c8b2d0e58201/kube-rbac-proxy/0.log"
Sep 14 05:28:39.391267 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.391241    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cslbk_f1554549-75e9-4d27-b4c5-c8b2d0e58201/init-textfile/0.log"
Sep 14 05:28:39.841731 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.841634    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vx2s4_38993dd4-9b2f-4abd-b9de-9332f1b6ac5b/prometheus-operator/0.log"
Sep 14 05:28:39.864564 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.864527    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vx2s4_38993dd4-9b2f-4abd-b9de-9332f1b6ac5b/kube-rbac-proxy/0.log"
Sep 14 05:28:39.935760 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.935708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7869cbc9fd-4tdhk_ee281523-5d85-4599-b09a-ac9a6dea4498/telemeter-client/0.log"
Sep 14 05:28:39.960774 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.960743    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7869cbc9fd-4tdhk_ee281523-5d85-4599-b09a-ac9a6dea4498/reload/0.log"
Sep 14 05:28:39.984209 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:39.984175    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7869cbc9fd-4tdhk_ee281523-5d85-4599-b09a-ac9a6dea4498/kube-rbac-proxy/0.log"
Sep 14 05:28:40.014449 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.014418    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/thanos-query/0.log"
Sep 14 05:28:40.035389 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.035335    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/kube-rbac-proxy-web/0.log"
Sep 14 05:28:40.059766 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.059722    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/kube-rbac-proxy/0.log"
Sep 14 05:28:40.083414 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.083382    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/prom-label-proxy/0.log"
Sep 14 05:28:40.112305 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.112231    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/kube-rbac-proxy-rules/0.log"
Sep 14 05:28:40.142108 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:40.142076    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-74b6cc799d-cqsm7_96a70088-ecb8-4a11-8c4d-32a79e686fb7/kube-rbac-proxy-metrics/0.log"
Sep 14 05:28:41.494234 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:41.494203    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-scrkt_2d000910-baed-4c0c-90a7-607e2b0f8664/networking-console-plugin/0.log"
Sep 14 05:28:42.082498 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.082457    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/2.log"
Sep 14 05:28:42.087333 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.087299    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-dn8r4_c016e1b5-cc4b-41f5-abe7-424d6a8023d7/console-operator/3.log"
Sep 14 05:28:42.481912 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.481878    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"]
Sep 14 05:28:42.482308 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.482295    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="454f9f2a-f35c-4658-bd10-5487c8751089" containerName="maas-api"
Sep 14 05:28:42.482355 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.482309    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="454f9f2a-f35c-4658-bd10-5487c8751089" containerName="maas-api"
Sep 14 05:28:42.482395 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.482381    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="454f9f2a-f35c-4658-bd10-5487c8751089" containerName="maas-api"
Sep 14 05:28:42.485855 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.485833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.488370 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.488347    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvbqd\"/\"kube-root-ca.crt\""
Sep 14 05:28:42.488541 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.488513    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvbqd\"/\"openshift-service-ca.crt\""
Sep 14 05:28:42.489270 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.489250    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rvbqd\"/\"default-dockercfg-s74lc\""
Sep 14 05:28:42.494141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.494120    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"]
Sep 14 05:28:42.656663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.656617    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-proc\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.656663 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.656666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-sys\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.657141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.656693    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-podres\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.657141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.656773    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4fbw\" (UniqueName: \"kubernetes.io/projected/edea8e4f-08a0-498e-a099-8979ac63ee8f-kube-api-access-w4fbw\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.657141 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.656809    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-lib-modules\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.697813 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.697767    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cfdd6d869-j8hf7_186ec657-2c10-4eec-a544-323a1b0b673c/console/0.log"
Sep 14 05:28:42.758263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-proc\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-sys\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758263 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-podres\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4fbw\" (UniqueName: \"kubernetes.io/projected/edea8e4f-08a0-498e-a099-8979ac63ee8f-kube-api-access-w4fbw\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-proc\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-lib-modules\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758336    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-sys\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758408    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-podres\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.758568 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.758411    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edea8e4f-08a0-498e-a099-8979ac63ee8f-lib-modules\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.769325 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.769293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4fbw\" (UniqueName: \"kubernetes.io/projected/edea8e4f-08a0-498e-a099-8979ac63ee8f-kube-api-access-w4fbw\") pod \"perf-node-gather-daemonset-fd6rq\" (UID: \"edea8e4f-08a0-498e-a099-8979ac63ee8f\") " pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:42.796460 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:42.796417    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:43.072140 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:43.072112    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"]
Sep 14 05:28:43.074473 ip-10-0-137-105 kubenswrapper[2571]: W0914 05:28:43.074444    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podedea8e4f_08a0_498e_a099_8979ac63ee8f.slice/crio-40d39224aa409f152c67075bcf2d998aa07a2ac2a6acad8bd777f25ba9f651c4 WatchSource:0}: Error finding container 40d39224aa409f152c67075bcf2d998aa07a2ac2a6acad8bd777f25ba9f651c4: Status 404 returned error can't find the container with id 40d39224aa409f152c67075bcf2d998aa07a2ac2a6acad8bd777f25ba9f651c4
Sep 14 05:28:43.675146 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:43.675093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq" event={"ID":"edea8e4f-08a0-498e-a099-8979ac63ee8f","Type":"ContainerStarted","Data":"ff8662f1b4222d8b8754dd7f979cae2138757a47ec64bfa62f88579b77c64ba7"}
Sep 14 05:28:43.675614 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:43.675303    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:43.675614 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:43.675337    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq" event={"ID":"edea8e4f-08a0-498e-a099-8979ac63ee8f","Type":"ContainerStarted","Data":"40d39224aa409f152c67075bcf2d998aa07a2ac2a6acad8bd777f25ba9f651c4"}
Sep 14 05:28:43.691908 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:43.691846    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq" podStartSLOduration=1.691829711 podStartE2EDuration="1.691829711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 05:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 05:28:43.69067439 +0000 UTC m=+2518.052626833" watchObservedRunningTime="2026-09-14 05:28:43.691829711 +0000 UTC m=+2518.053782154"
Sep 14 05:28:44.659489 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:44.659461    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zw87b_f3bebbe5-347c-46df-b90e-542a9c8882db/dns/0.log"
Sep 14 05:28:44.680433 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:44.680411    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zw87b_f3bebbe5-347c-46df-b90e-542a9c8882db/kube-rbac-proxy/0.log"
Sep 14 05:28:44.709324 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:44.709285    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnm2q_df98b66d-882d-4433-a8a2-170911fd83be/dns-node-resolver/0.log"
Sep 14 05:28:45.369614 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:45.369567    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7zh6h_804bd645-d44d-436f-afde-bc038ec57fc1/node-ca/0.log"
Sep 14 05:28:46.714652 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:46.714624    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96745c74f-dttph_82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04/kube-auth-proxy/0.log"
Sep 14 05:28:46.807456 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:46.807424    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gtt2g_cda02c30-fd51-4775-9881-11d9b3493245/payload-pre-processing/0.log"
Sep 14 05:28:47.817029 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:47.816991    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-blwf5_fc32cf52-6d5e-489a-9589-3cc91fe9dbd2/serve-healthcheck-canary/0.log"
Sep 14 05:28:48.438382 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:48.438347    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55sbp_5369a29b-707b-42ed-a29c-f34c71fce8f8/kube-rbac-proxy/0.log"
Sep 14 05:28:48.460064 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:48.460031    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55sbp_5369a29b-707b-42ed-a29c-f34c71fce8f8/exporter/0.log"
Sep 14 05:28:48.482329 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:48.482295    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-55sbp_5369a29b-707b-42ed-a29c-f34c71fce8f8/extractor/0.log"
Sep 14 05:28:49.691789 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:49.691753    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rvbqd/perf-node-gather-daemonset-fd6rq"
Sep 14 05:28:50.637071 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:50.637044    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5jl2t_50c9c4c1-a7b2-453a-9c5d-69497bc0b5fb/manager/0.log"
Sep 14 05:28:50.852518 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:50.852484    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ttvrr_c16eebb5-a63b-4538-bf06-045b63b7d9db/manager/0.log"
Sep 14 05:28:50.901647 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:50.901564    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c44665c5c-5s4tn_0cb003ca-953e-47b1-85ec-7479a880812a/manager/0.log"
Sep 14 05:28:50.922334 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:50.922312    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5wwgd_8ef3c739-010f-48a7-a2a4-318b70c37d4d/server/0.log"
Sep 14 05:28:51.084867 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:51.084840    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b6456dc5-p42j6_282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a/manager/0.log"
Sep 14 05:28:51.102757 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:51.102725    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b6456dc5-p42j6_282ec9ea-0c92-4ed7-b8b5-7c7af2c34f9a/copy-manifests/0.log"
Sep 14 05:28:52.724114 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:52.724082    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-85trj_e2ca4591-bf29-4a32-9516-ffa8c806b289/openshift-lws-operator/0.log"
Sep 14 05:28:53.223281 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:53.223251    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96745c74f-dttph_82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04/kube-auth-proxy/0.log"
Sep 14 05:28:53.282135 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:53.282085    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gtt2g_cda02c30-fd51-4775-9881-11d9b3493245/payload-pre-processing/0.log"
Sep 14 05:28:55.241255 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:55.241221    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-96745c74f-dttph_82aea7cf-0a7b-4c24-ad00-c48a5bd1bf04/kube-auth-proxy/0.log"
Sep 14 05:28:55.308043 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:55.308009    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gtt2g_cda02c30-fd51-4775-9881-11d9b3493245/payload-pre-processing/0.log"
Sep 14 05:28:58.239458 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:58.239413    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-fsrgh_27a1027d-4dc2-4f57-a251-57eb0b9e95b3/kube-storage-version-migrator-operator/1.log"
Sep 14 05:28:58.240408 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:58.240359    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-fsrgh_27a1027d-4dc2-4f57-a251-57eb0b9e95b3/kube-storage-version-migrator-operator/0.log"
Sep 14 05:28:59.309987 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.309958    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/kube-multus-additional-cni-plugins/0.log"
Sep 14 05:28:59.331095 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.331019    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/egress-router-binary-copy/0.log"
Sep 14 05:28:59.350579 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.350547    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/cni-plugins/0.log"
Sep 14 05:28:59.372770 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.372742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/bond-cni-plugin/0.log"
Sep 14 05:28:59.393161 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.393129    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/routeoverride-cni/0.log"
Sep 14 05:28:59.416208 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.416182    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/whereabouts-cni-bincopy/0.log"
Sep 14 05:28:59.437098 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.437066    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4m8tl_2641c440-5495-414d-8912-829662eccbc4/whereabouts-cni/0.log"
Sep 14 05:28:59.872290 ip-10-0-137-105 kubenswrapper[2571]: I0914 05:28:59.872253    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qm95s_600898a9-bb9b-4b00-ac5e-ce2fef5dde22/kube-multus/0.log"