Sep 11 03:38:26.109440 ip-10-0-139-129 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 03:38:26.109454 ip-10-0-139-129 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 03:38:26.109463 ip-10-0-139-129 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 03:38:26.109817 ip-10-0-139-129 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 03:38:36.138468 ip-10-0-139-129 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 03:38:36.138486 ip-10-0-139-129 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b06bcf885dd64d76b6cb5e1368c32a60 --
Sep 11 03:40:10.251105 ip-10-0-139-129 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 03:40:10.676035 ip-10-0-139-129 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 03:40:10.676035 ip-10-0-139-129 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 03:40:10.676035 ip-10-0-139-129 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 03:40:10.676035 ip-10-0-139-129 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 03:40:10.676035 ip-10-0-139-129 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 03:40:10.678526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.678431    2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 03:40:10.685831 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685781    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.685948 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685931    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.685948 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685947    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685951    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685957    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685960    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685963    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685966    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685969    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685972    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685975    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685978    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685981    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685985    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685988    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685990    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685993    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685996    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.685999    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686002    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686004    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686007    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.686016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686010    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686013    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686015    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686018    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686021    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686025    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686033    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686040    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686043    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686046    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686049    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686052    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686055    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686057    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686060    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686063    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686066    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686070    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686073    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686076    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.686493 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686078    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686082    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686087    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686090    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686092    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686096    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686101    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686104    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686107    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686110    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686112    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686115    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686118    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686120    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686124    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686127    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686130    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686133    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686136    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.686993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686142    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686146    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686149    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686152    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686155    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686157    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686160    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686163    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686166    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686169    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686171    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686174    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686176    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686179    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686181    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686184    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686187    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686189    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686192    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686195    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.687458 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686198    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686200    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686203    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686206    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686209    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686211    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686214    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686217    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686220    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686223    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686226    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686229    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686232    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686234    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686237    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686239    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686242    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686245    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686976    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686984    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.687971 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686987    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686989    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686992    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.686995    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687000    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687004    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687007    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687011    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687014    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687016    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687019    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687022    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687025    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687028    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687031    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687035    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687038    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687040    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687043    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687047    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.688460 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687050    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687053    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687056    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687059    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687061    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687064    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687067    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687070    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687073    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687075    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687079    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687081    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687084    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687086    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687089    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687092    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687094    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687097    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687100    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687102    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.688980 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687105    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687107    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687110    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687112    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687115    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687117    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687120    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687122    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687126    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687128    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687131    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687134    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687137    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687139    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687142    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687145    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687148    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687150    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687153    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687155    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.689481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687158    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687160    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687163    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687165    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687168    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687171    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687173    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687176    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687179    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687182    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687184    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687187    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687189    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687192    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687195    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687198    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687201    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687203    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687207    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.690006 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687210    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687213    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687217    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687220    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687223    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687226    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687228    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687231    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687233    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687236    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687240    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687242    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687245    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687249    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687252    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687255    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687257    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687260    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687702    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.690470 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687709    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687712    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687715    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687718    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687721    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687724    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687727    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687730    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687733    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687736    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687738    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687741    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687744    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687747    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687749    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687755    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687759    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687761    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687764    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687766    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.690962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687769    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687771    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687774    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687776    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687779    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687781    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687784    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687786    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687789    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687791    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687794    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687796    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687799    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687802    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687804    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687809    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687825    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687832    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687835    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687837    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.691481 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687840    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687843    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687846    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687848    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687851    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687854    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687856    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687859    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687869    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687873    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687876    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687878    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687881    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687883    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687886    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687889    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687891    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687894    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687896    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.691997 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687899    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687905    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687907    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687910    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687913    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687915    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687918    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687921    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687923    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687926    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687930    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687932    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687935    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687938    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687940    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687943    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687946    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687948    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687951    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687953    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.692465 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687956    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687958    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687964    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687967    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687969    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687972    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687974    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687977    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687980    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687983    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687985    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687989    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687992    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687996    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.687999    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688002    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688005    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688008    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688010    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688085    2553 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 03:40:10.692993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688118    2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688128    2553 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688133    2553 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688137    2553 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688141    2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688145    2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688150    2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688153    2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688156    2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688159    2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688162    2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688165    2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688168    2553 flags.go:64] FLAG: --cgroup-root=""
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688171    2553 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688174    2553 flags.go:64] FLAG: --client-ca-file=""
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688178    2553 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688181    2553 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688188    2553 flags.go:64] FLAG: --cluster-domain=""
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688191    2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688194    2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688196    2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688200    2553 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688204    2553 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 03:40:10.693473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688206    2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688209    2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688214    2553 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688217    2553 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688220    2553 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688223    2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688226    2553 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688228    2553 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688232    2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688235    2553 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688238    2553 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688241    2553 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688244    2553 flags.go:64] FLAG: --enable-server="true"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688247    2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688252    2553 flags.go:64] FLAG: --event-burst="100"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688255    2553 flags.go:64] FLAG: --event-qps="50"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688258    2553 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688262    2553 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688264    2553 flags.go:64] FLAG: --eviction-hard=""
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688268    2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688271    2553 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688274    2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688277    2553 flags.go:64] FLAG: --eviction-soft=""
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688280    2553 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688283    2553 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 03:40:10.694044 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688287    2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688291    2553 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688294    2553 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688296    2553 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688299    2553 flags.go:64] FLAG: --feature-gates=""
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688303    2553 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688306    2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688309    2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688312    2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688316    2553 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688319    2553 flags.go:64] FLAG: --help="false"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688322    2553 flags.go:64] FLAG: --hostname-override="ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688326    2553 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688328    2553 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688331    2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688334    2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688338    2553 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688341    2553 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688344    2553 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688347    2553 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688350    2553 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688353    2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688356    2553 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688359    2553 flags.go:64] FLAG: --kube-reserved=""
Sep 11 03:40:10.694679 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688362    2553 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688364    2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688367    2553 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688370    2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688373    2553 flags.go:64] FLAG: --lock-file=""
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688376    2553 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688379    2553 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688382    2553 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688387    2553 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688393    2553 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688396    2553 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688399    2553 flags.go:64] FLAG: --logging-format="text"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688402    2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688405    2553 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688408    2553 flags.go:64] FLAG: --manifest-url=""
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688411    2553 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688416    2553 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688420    2553 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688425    2553 flags.go:64] FLAG: --max-pods="110"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688428    2553 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688431    2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688434    2553 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688437    2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688440    2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688443    2553 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 03:40:10.695284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688446    2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688452    2553 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688455    2553 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688459    2553 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688462    2553 flags.go:64] FLAG: --pod-cidr=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688464    2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688470    2553 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688472    2553 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688476    2553 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688479    2553 flags.go:64] FLAG: --port="10250"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688482    2553 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688485    2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af62657c9e31ca6a"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688488    2553 flags.go:64] FLAG: --qos-reserved=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688491    2553 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688494    2553 flags.go:64] FLAG: --register-node="true"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688497    2553 flags.go:64] FLAG: --register-with-taints=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688500    2553 flags.go:64] FLAG: --registry-burst="10"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688504    2553 flags.go:64] FLAG: --registry-qps="5"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688507    2553 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688510    2553 flags.go:64] FLAG: --reserved-memory=""
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688514    2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688517    2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688520    2553 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688523    2553 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688527    2553 flags.go:64] FLAG: --runonce="false"
Sep 11 03:40:10.695924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688530    2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688533    2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688536    2553 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688539    2553 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688542    2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688545    2553 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688548    2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688551    2553 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688554    2553 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688559    2553 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688562    2553 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688564    2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688567    2553 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688570    2553 flags.go:64] FLAG: --system-cgroups=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688573    2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688579    2553 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688581    2553 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688584    2553 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688588    2553 flags.go:64] FLAG: --tls-min-version=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688591    2553 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688593    2553 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688596    2553 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688599    2553 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688602    2553 flags.go:64] FLAG: --v="2"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688606    2553 flags.go:64] FLAG: --version="false"
Sep 11 03:40:10.696606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688611    2553 flags.go:64] FLAG: --vmodule=""
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688615    2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.688618    2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688740    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688744    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688747    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688752    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688755    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688758    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688761    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688764    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688766    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688769    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688771    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688774    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688777    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688781    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688784    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688787    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688790    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.697235 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688793    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688795    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688798    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688801    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688804    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688807    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688809    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688832    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688836    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688840    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688844    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688848    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688853    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688858    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688861    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688865    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688868    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688871    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688875    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688878    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.697734 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688881    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688884    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688887    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688889    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688892    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688895    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688897    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688900    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688904    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688907    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688911    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688915    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688918    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688921    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688924    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688926    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688929    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688932    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688934    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.698316 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688937    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688940    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688942    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688945    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688948    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688950    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688953    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688956    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688959    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688961    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688964    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688967    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688970    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688973    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688975    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688978    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688981    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688983    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688986    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688989    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.698798 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688992    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.688997    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689001    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689005    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689008    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689011    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689014    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689016    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689020    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689024    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689027    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689030    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689033    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689035    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689038    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689040    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689043    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689046    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689049    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689051    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.699302 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689055    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689058    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.689061    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.689977    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.697680    2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.697698    2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.698542    2553 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.698554    2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698612    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698618    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698621    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698625    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698628    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698631    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698633    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.699810 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698636    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698639    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698642    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698646    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698650    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698654    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698657    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698660    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698663    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698665    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698668    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698670    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698673    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698675    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698678    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698681    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698684    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698686    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698689    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698692    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.700234 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698694    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698697    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698701    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698704    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698707    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698710    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698713    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698715    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698718    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698721    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698723    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698726    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698728    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698731    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698734    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698737    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698739    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698742    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698744    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698747    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.700721 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698750    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698752    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698755    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698758    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698762    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698765    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698768    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698771    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698774    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698777    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698780    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698783    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698787    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698790    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698793    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698796    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698799    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698802    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698805    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.701228 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698808    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698821    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698824    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698827    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698829    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698832    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698835    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698839    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698841    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698844    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698846    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698849    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698852    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698854    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698857    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698859    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698862    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698865    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698867    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698870    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.701694 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698872    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698875    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698878    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698881    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698884    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698887    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698890    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698892    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698895    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698898    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698902    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698906    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.698909    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.698914    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699031    2553 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 03:40:10.702265 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699035    2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699038    2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699041    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699044    2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699047    2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699050    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699052    2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699055    2553 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699057    2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699060    2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699063    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699065    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699068    2553 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699071    2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699074    2553 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699076    2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699079    2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699081    2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699084    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699086    2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 03:40:10.702651 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699089    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699092    2553 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699095    2553 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699098    2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699100    2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699103    2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699106    2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699110    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699113    2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699116    2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699119    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699121    2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699124    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699126    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699129    2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699131    2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699134    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699136    2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699139    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 03:40:10.703169 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699142    2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699144    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699147    2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699149    2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699152    2553 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699154    2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699157    2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699159    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699162    2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699164    2553 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699167    2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699169    2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699172    2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699176    2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699180    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699184    2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699187    2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699190    2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699193    2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 03:40:10.703643 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699196    2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699198    2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699202    2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699205    2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699208    2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699210    2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699213    2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699216    2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699219    2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699221    2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699224    2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699226    2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699229    2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699231    2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699234    2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699237    2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699240    2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699242    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699245    2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699248    2553 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 03:40:10.704115 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699250    2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699253    2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699255    2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699258    2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699260    2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699263    2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699265    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699268    2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699271    2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699274    2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699276    2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699279    2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699281    2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699284    2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699286    2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699289    2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699292    2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699294    2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699297    2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 03:40:10.704597 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:10.699299    2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.699305    2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.699420    2553 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.701842    2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.702896    2553 server.go:1019] "Starting client certificate rotation"
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.702996    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 03:40:10.705076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.703030    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 03:40:10.705799 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.705785    2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 03:40:10.707003 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.706985    2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 03:40:10.718717 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.718692    2553 log.go:25] "Validated CRI v1 runtime API"
Sep 11 03:40:10.722012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.721996    2553 log.go:25] "Validated CRI v1 image API"
Sep 11 03:40:10.723954 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.723933    2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 03:40:10.727476 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.727454    2553 fs.go:135] Filesystem UUIDs: map[365d9328-d2e9-48f0-9f00-3aa3e36da9df:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c763ab50-bd28-4ca9-b705-63271279a5e6:/dev/nvme0n1p3]
Sep 11 03:40:10.727584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.727474    2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 03:40:10.731446 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.731427    2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 03:40:10.732993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.732865    2553 manager.go:216] Machine: {Timestamp:2026-09-11 03:40:10.731392776 +0000 UTC m=+0.374182928 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101995 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec243732e5bf78b2e87b1d5118ca844e SystemUUID:ec243732-e5bf-78b2-e87b-1d5118ca844e BootID:b06bcf88-5dd6-4d76-b6cb-5e1368c32a60 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:78:ca:b4:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:78:ca:b4:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:ab:e0:98:a4:cc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 03:40:10.732993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.732987    2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 03:40:10.733209 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.733109    2553 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 03:40:10.736539 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.736510    2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 03:40:10.736702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.736542    2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 03:40:10.737543 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.737530    2553 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 03:40:10.737600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.737550    2553 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 03:40:10.737600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.737561    2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 03:40:10.738555 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.738541    2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 03:40:10.739707 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.739686    2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 03:40:10.743061 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743047    2553 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 03:40:10.743195 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743184    2553 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 03:40:10.743249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743224    2553 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 03:40:10.743249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743238    2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 03:40:10.743331 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743262    2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 03:40:10.743331 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743282    2553 kubelet.go:398] "Adding apiserver pod source"
Sep 11 03:40:10.743331 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743294    2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 03:40:10.743708 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743694    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 03:40:10.743780 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.743719    2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 03:40:10.746364 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.746345    2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 03:40:10.749246 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.749232    2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 03:40:10.749307 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.749261    2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 03:40:10.751285 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751263    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751289    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751304    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751316    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751336    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751346    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751369    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751382    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751395    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 03:40:10.751419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751405    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 03:40:10.751744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751439    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 03:40:10.751744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.751453    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 03:40:10.751744 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.751617    2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 03:40:10.751961 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.751939    2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 03:40:10.752435 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.752423    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 03:40:10.752529 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.752439    2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 03:40:10.754084 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.754044    2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 03:40:10.756503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.756487    2553 server.go:1268] "Started kubelet"
Sep 11 03:40:10.756592 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.756546    2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 03:40:10.756677 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.756622    2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 03:40:10.756733 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.756709    2553 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 03:40:10.757717 ip-10-0-139-129 systemd[1]: Started Kubernetes Kubelet.
Sep 11 03:40:10.758923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.758910    2553 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 03:40:10.759192 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.759175    2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 03:40:10.763148 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.763129    2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 03:40:10.766234 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.766208    2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 03:40:10.766908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.766886    2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 03:40:10.767043 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.767020    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:10.767842 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.767784    2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 03:40:10.768017 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.767802    2553 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 03:40:10.768071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.768022    2553 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 03:40:10.768327 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.768303    2553 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 03:40:10.768327 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.768326    2553 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 03:40:10.769615 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.769598    2553 factory.go:55] Registering systemd factory
Sep 11 03:40:10.769714 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.769705    2553 factory.go:223] Registration of the systemd container factory successfully
Sep 11 03:40:10.770054 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.770033    2553 factory.go:153] Registering CRI-O factory
Sep 11 03:40:10.770124 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.770059    2553 factory.go:223] Registration of the crio container factory successfully
Sep 11 03:40:10.770176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.770131    2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 03:40:10.770176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.770150    2553 factory.go:103] Registering Raw factory
Sep 11 03:40:10.770176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.770165    2553 manager.go:1196] Started watching for new ooms in manager
Sep 11 03:40:10.771558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.771536    2553 manager.go:349] Starting recovery of all containers
Sep 11 03:40:10.771728 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.771608    2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 03:40:10.771728 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.771615    2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 03:40:10.772893 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.771641    2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-129.ec2.internal.18d42729677ef765  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-129.ec2.internal,UID:ip-10-0-139-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-129.ec2.internal,},FirstTimestamp:2026-09-11 03:40:10.756446053 +0000 UTC m=+0.399236206,LastTimestamp:2026-09-11 03:40:10.756446053 +0000 UTC m=+0.399236206,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-129.ec2.internal,}"
Sep 11 03:40:10.776789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.776768    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r5h9l"
Sep 11 03:40:10.783741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.783552    2553 manager.go:354] Recovery completed
Sep 11 03:40:10.783945 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.783924    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r5h9l"
Sep 11 03:40:10.788272 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.788258    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.790007 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.789988    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.790104 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790021    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.790104 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790036    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.790739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790725    2553 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 03:40:10.790739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790737    2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 03:40:10.790895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790750    2553 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 03:40:10.790895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.790802    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.791613 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791597    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.791689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791621    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.791689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791640    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.791689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791667    2553 policy_none.go:49] "None policy: Start"
Sep 11 03:40:10.791689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791685    2553 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 03:40:10.791887 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.791699    2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 03:40:10.791887 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.791719    2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-129.ec2.internal.18d42729697f0a2c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-129.ec2.internal,UID:ip-10-0-139-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-129.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-129.ec2.internal,},FirstTimestamp:2026-09-11 03:40:10.790005292 +0000 UTC m=+0.432795445,LastTimestamp:2026-09-11 03:40:10.790005292 +0000 UTC m=+0.432795445,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-129.ec2.internal,}"
Sep 11 03:40:10.792617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.792606    2553 policy_none.go:47] "Start"
Sep 11 03:40:10.830670 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.830607    2553 manager.go:337] "Starting Device Plugin manager"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.830692    2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.830706    2553 server.go:87] "Starting device plugin registration server"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.831252    2553 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.831267    2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.831366    2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.831453    2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.831462    2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.832171    2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 03:40:10.833023 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.832294    2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:10.864175 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.864130    2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 03:40:10.865534 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.865507    2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 03:40:10.865623 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.865574    2553 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 03:40:10.865623 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.865601    2553 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 03:40:10.865714 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.865650    2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 03:40:10.869244 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.869223    2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 03:40:10.931775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.931658    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.933125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.933097    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.933237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.933144    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.933237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.933160    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.933237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.933197    2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.942358 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.942339    2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.942452 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.942364    2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-129.ec2.internal\": node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:10.965071 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.965039    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:10.966062 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.966027    2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"]
Sep 11 03:40:10.966122 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.966113    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.967085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.967062    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.967196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.967095    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.967196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.967107    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.968289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.968276    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.968424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.968402    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.968461 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.968435    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.969023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969001    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.969023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969015    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.969152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969030    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.969152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969035    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.969152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969040    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.969152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969045    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.969312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969193    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.969312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.969220    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.970125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.970113    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:10.970170 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.970135    2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 03:40:10.970790 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.970772    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 03:40:10.970887 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.970803    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 03:40:10.970887 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:10.970829    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeHasSufficientPID"
Sep 11 03:40:10.998454 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:10.998434    2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-129.ec2.internal\" not found" node="ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.002871 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.002856    2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-129.ec2.internal\" not found" node="ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.065294 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.065267    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.069604 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.069589    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.069659 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.069611    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.069659 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.069631    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0479989c71536ba20c64f751ffba4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-129.ec2.internal\" (UID: \"85b0479989c71536ba20c64f751ffba4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.069727 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.069690    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.069727 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.069704    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5493fdbc7f8db30fc5476d18f9a9b83-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal\" (UID: \"f5493fdbc7f8db30fc5476d18f9a9b83\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.165821 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.165777    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.170117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.170099    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0479989c71536ba20c64f751ffba4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-129.ec2.internal\" (UID: \"85b0479989c71536ba20c64f751ffba4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.170164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.170151    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/85b0479989c71536ba20c64f751ffba4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-129.ec2.internal\" (UID: \"85b0479989c71536ba20c64f751ffba4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.266587 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.266516    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.300975 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.300948    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.305584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.305570    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:11.366848 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.366798    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.467249 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.467215    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.567587 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.567514    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.667853 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.667805    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.703152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.703131    2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 03:40:11.703514 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.703255    2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 03:40:11.768026 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.767996    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.788892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.788857    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 03:35:10 +0000 UTC" deadline="2028-05-01 05:05:44.265435763 +0000 UTC"
Sep 11 03:40:11.788977 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.788892    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14353h25m32.476547164s"
Sep 11 03:40:11.797890 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:11.797858    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5493fdbc7f8db30fc5476d18f9a9b83.slice/crio-7fb93241e067967e1569ea7344e00a73e6907807f16b9bfdc8e8feee200040a1 WatchSource:0}: Error finding container 7fb93241e067967e1569ea7344e00a73e6907807f16b9bfdc8e8feee200040a1: Status 404 returned error can't find the container with id 7fb93241e067967e1569ea7344e00a73e6907807f16b9bfdc8e8feee200040a1
Sep 11 03:40:11.802449 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.802434    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 03:40:11.806766 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:11.806740    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b0479989c71536ba20c64f751ffba4.slice/crio-5c1b8dd13111fbe01ed2c153c785e7361c5f616f2163a2619791479cd1181cbc WatchSource:0}: Error finding container 5c1b8dd13111fbe01ed2c153c785e7361c5f616f2163a2619791479cd1181cbc: Status 404 returned error can't find the container with id 5c1b8dd13111fbe01ed2c153c785e7361c5f616f2163a2619791479cd1181cbc
Sep 11 03:40:11.868191 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.868124    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:11.869661 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.869611    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal" event={"ID":"85b0479989c71536ba20c64f751ffba4","Type":"ContainerStarted","Data":"5c1b8dd13111fbe01ed2c153c785e7361c5f616f2163a2619791479cd1181cbc"}
Sep 11 03:40:11.870524 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:11.870505    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal" event={"ID":"f5493fdbc7f8db30fc5476d18f9a9b83","Type":"ContainerStarted","Data":"7fb93241e067967e1569ea7344e00a73e6907807f16b9bfdc8e8feee200040a1"}
Sep 11 03:40:11.968892 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:11.968857    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:12.001503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.001475    2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 03:40:12.016090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.016070    2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 03:40:12.069084 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.069057    2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-129.ec2.internal\" not found"
Sep 11 03:40:12.129142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.129078    2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 03:40:12.168120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.168093    2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:12.180218 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.180199    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 03:40:12.181119 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.181096    2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal"
Sep 11 03:40:12.187267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.187247    2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 03:40:12.541357 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.541280    2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 03:40:12.744735 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.744696    2553 apiserver.go:52] "Watching apiserver"
Sep 11 03:40:12.751511 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.751484    2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 03:40:12.751916 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.751887    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-lh9v2","openshift-dns/node-resolver-7c4sh","openshift-multus/multus-additional-cni-plugins-5ndjs","openshift-multus/network-metrics-daemon-2wqbk","openshift-network-diagnostics/network-check-target-pdssj","openshift-network-operator/iptables-alerter-vg9pw","openshift-ovn-kubernetes/ovnkube-node-86xfr","kube-system/global-pull-secret-syncer-s5gzw","kube-system/konnectivity-agent-9d52k","kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp","openshift-cluster-node-tuning-operator/tuned-cbdwp","openshift-image-registry/node-ca-jgkcv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal"]
Sep 11 03:40:12.753809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.753782    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.754851 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.754824    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.756089 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.756066    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.756719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.756698    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 03:40:12.758564 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.758091    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 03:40:12.758564 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.758396    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fmqrc\""
Sep 11 03:40:12.758720 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.758707    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.759412 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.758795    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:12.759412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.758892    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 03:40:12.759412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.758994    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 03:40:12.759412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759195    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.759412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759210    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 03:40:12.759685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759440    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.759685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759503    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.759685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759518    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5vd9x\""
Sep 11 03:40:12.759685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759539    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 03:40:12.760382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759948    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.760382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.759992    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 03:40:12.760382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.760223    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m74qq\""
Sep 11 03:40:12.760584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.760399    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 03:40:12.760771 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.760753    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 03:40:12.760889 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.760803    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.761729 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.761713    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.761996 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.761980    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 03:40:12.762344 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.762330    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vx8ml\""
Sep 11 03:40:12.763095 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.763079    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.763883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.763862    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.764875 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.764851    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.764977 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.764936    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5c4gv\""
Sep 11 03:40:12.764977 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.764974    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.765378    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.765469    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.765519    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.765531    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.765578    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.765830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.765662    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jshf8\""
Sep 11 03:40:12.766690 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.766674    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.767006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.766988    2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 03:40:12.768208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.768188    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.768913 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.768894    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.769001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.768931    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4qfc6\""
Sep 11 03:40:12.769176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.769157    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.769717 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.769698    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.770605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.770588    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 03:40:12.770705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.770611    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mxbvh\""
Sep 11 03:40:12.770705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.770622    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.770705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.770726    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.770951 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.770900    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.770996 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.770945    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:12.772216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.772200    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 03:40:12.772447 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.772422    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 03:40:12.772447 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.772438    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mqgmz\""
Sep 11 03:40:12.776476 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.776458    2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 03:40:12.776974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.776957    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-systemd-units\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777065 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.776984    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-config\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777065 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777005    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.777065 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777020    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-modprobe-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.777065 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777039    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc10e86-f575-4533-94a3-911036908965-konnectivity-ca\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777080    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qb72\" (UniqueName: \"kubernetes.io/projected/b54926db-c238-4f65-8827-207fd6181548-kube-api-access-8qb72\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777095    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-sys\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777113    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxx6\" (UniqueName: \"kubernetes.io/projected/1da582df-6d44-4b9a-a39f-a903d23c23b2-kube-api-access-snxx6\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777138    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-systemd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777162    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716a74a8-e82d-4a3b-8716-987ee521f598-ovn-node-metrics-cert\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777189    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-etc-selinux\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.777223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777211    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-tuned\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777227    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-tmp\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777245    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-k8s-cni-cncf-io\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777277    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-hostroot\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777295    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-daemon-config\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777320    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-cnibin\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777336    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777354    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-kubelet\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777379    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-script-lib\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777408    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-bin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777430    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-multus-certs\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777444    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-etc-kubernetes\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777459    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-netns\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777474    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-bin\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777489    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.777502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777503    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-registration-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777526    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-kubernetes\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777546    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-run\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777560    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777577    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c64109e2-8108-4841-bac6-5117f7bf60a1-tmp-dir\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777591    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-sys-fs\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777605    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-lib-modules\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777629    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-var-lib-kubelet\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777655    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-system-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777675    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dba23ae-c972-4567-9926-d61a07546e4d-host\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777696    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dba23ae-c972-4567-9926-d61a07546e4d-serviceca\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777717    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-node-log\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777732    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777748    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-device-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777763    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmn99\" (UniqueName: \"kubernetes.io/projected/7dba23ae-c972-4567-9926-d61a07546e4d-kube-api-access-hmn99\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777782    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-iptables-alerter-script\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777806    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-cni-binary-copy\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777847    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-conf-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777879    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-var-lib-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777904    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-ovn\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777925    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c64109e2-8108-4841-bac6-5117f7bf60a1-hosts-file\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777954    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbq5s\" (UniqueName: \"kubernetes.io/projected/c64109e2-8108-4841-bac6-5117f7bf60a1-kube-api-access-rbq5s\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777969    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-host\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.777988    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqwtq\" (UniqueName: \"kubernetes.io/projected/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-kube-api-access-kqwtq\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778003    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-cnibin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778016    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-slash\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778031    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-conf\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778044    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-multus\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778060    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-kubelet\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778081    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778104    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778126    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778149    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-socket-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.778807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778172    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778187    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-netd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778201    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-env-overrides\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778215    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2wg\" (UniqueName: \"kubernetes.io/projected/716a74a8-e82d-4a3b-8716-987ee521f598-kube-api-access-2q2wg\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778228    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-systemd\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778246    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-socket-dir-parent\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778275    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsj8\" (UniqueName: \"kubernetes.io/projected/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-kube-api-access-gnsj8\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778295    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcn88\" (UniqueName: \"kubernetes.io/projected/96433f43-d834-4c3f-8d92-8cad28968754-kube-api-access-gcn88\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778313    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc10e86-f575-4533-94a3-911036908965-agent-certs\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778374    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-netns\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778401    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-host-slash\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778419    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778447    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778472    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs2l5\" (UniqueName: \"kubernetes.io/projected/6e362fef-5048-4204-92c4-c79c8206b72c-kube-api-access-rs2l5\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778512    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-log-socket\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778651    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-system-cni-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.779582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778927    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysconfig\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.780373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.778969    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-etc-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.780373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.779024    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.802098 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.802075    2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xcc2q"
Sep 11 03:40:12.808918 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.808900    2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xcc2q"
Sep 11 03:40:12.869012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.868980    2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 03:40:12.879584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879547    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-systemd-units\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879595    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-config\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879614    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-systemd-units\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879637    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-modprobe-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879693    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc10e86-f575-4533-94a3-911036908965-konnectivity-ca\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.879745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879726    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879753    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qb72\" (UniqueName: \"kubernetes.io/projected/b54926db-c238-4f65-8827-207fd6181548-kube-api-access-8qb72\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879760    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-modprobe-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879769    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-sys\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879800    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxx6\" (UniqueName: \"kubernetes.io/projected/1da582df-6d44-4b9a-a39f-a903d23c23b2-kube-api-access-snxx6\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879826    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-sys\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.879799    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879842    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-systemd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879866    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716a74a8-e82d-4a3b-8716-987ee521f598-ovn-node-metrics-cert\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879890    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-etc-selinux\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.879913    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-tuned\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.880071 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.880055    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:13.380000656 +0000 UTC m=+3.022790820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880270    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-config\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880274    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-systemd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880322    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-tmp\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880351    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-etc-selinux\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880363    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-k8s-cni-cncf-io\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880379    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-hostroot\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880416    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-daemon-config\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880461    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-cnibin\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880489    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880497    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-hostroot\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880504    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4bc10e86-f575-4533-94a3-911036908965-konnectivity-ca\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880518    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-kubelet\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880543    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-cnibin\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880562    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-kubelet\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.880573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880563    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-script-lib\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880607    2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880616    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880584    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-k8s-cni-cncf-io\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880673    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-bin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880721    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-multus-certs\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880747    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-etc-kubernetes\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880795    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-multus-certs\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880877    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-bin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880878    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-etc-kubernetes\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880904    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-netns\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880938    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-bin\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880964    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.880989    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-registration-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881013    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-kubernetes\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881018    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-daemon-config\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881037    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-run\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881050    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-ovnkube-script-lib\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.881314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881062    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881082    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-netns\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881095    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c64109e2-8108-4841-bac6-5117f7bf60a1-tmp-dir\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881120    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-sys-fs\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881124    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881121    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-bin\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881176    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-kubernetes\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881183    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-sys-fs\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881233    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-registration-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881262    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-lib-modules\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881288    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-var-lib-kubelet\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881311    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-system-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881334    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dba23ae-c972-4567-9926-d61a07546e4d-host\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881356    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dba23ae-c972-4567-9926-d61a07546e4d-serviceca\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881380    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-node-log\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881385    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c64109e2-8108-4841-bac6-5117f7bf60a1-tmp-dir\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881404    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881409    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-lib-modules\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.882164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881456    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-device-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881477    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-system-cni-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881483    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmn99\" (UniqueName: \"kubernetes.io/projected/7dba23ae-c972-4567-9926-d61a07546e4d-kube-api-access-hmn99\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881519    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dba23ae-c972-4567-9926-d61a07546e4d-host\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881572    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-iptables-alerter-script\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881599    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-dbus\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881625    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-cni-binary-copy\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881646    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-conf-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881674    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-var-lib-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881699    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-ovn\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881723    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c64109e2-8108-4841-bac6-5117f7bf60a1-hosts-file\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881741    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-binary-copy\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881748    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbq5s\" (UniqueName: \"kubernetes.io/projected/c64109e2-8108-4841-bac6-5117f7bf60a1-kube-api-access-rbq5s\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881774    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-host\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881798    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqwtq\" (UniqueName: \"kubernetes.io/projected/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-kube-api-access-kqwtq\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881841    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-cnibin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881866    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-slash\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881897    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-conf\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881921    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dba23ae-c972-4567-9926-d61a07546e4d-serviceca\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881935    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-multus\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881974    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-kubelet\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.881981    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-cni-multus\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882014    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882019    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-var-lib-kubelet\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882040    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882069    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882083    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-device-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882094    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-socket-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882104    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-run\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882122    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882130    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-ovn\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882139    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-var-lib-kubelet\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882148    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-netd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882174    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-env-overrides\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882181    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-run-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.883839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882201    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2wg\" (UniqueName: \"kubernetes.io/projected/716a74a8-e82d-4a3b-8716-987ee521f598-kube-api-access-2q2wg\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882227    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-systemd\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882235    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c64109e2-8108-4841-bac6-5117f7bf60a1-hosts-file\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882270    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-socket-dir-parent\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882298    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsj8\" (UniqueName: \"kubernetes.io/projected/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-kube-api-access-gnsj8\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882323    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-kubelet-config\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882349    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcn88\" (UniqueName: \"kubernetes.io/projected/96433f43-d834-4c3f-8d92-8cad28968754-kube-api-access-gcn88\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882372    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc10e86-f575-4533-94a3-911036908965-agent-certs\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882396    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-netns\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882438    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-host-slash\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882464    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882490    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882515    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs2l5\" (UniqueName: \"kubernetes.io/projected/6e362fef-5048-4204-92c4-c79c8206b72c-kube-api-access-rs2l5\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882558    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-log-socket\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882582    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-system-cni-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882600    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1da582df-6d44-4b9a-a39f-a903d23c23b2-cni-binary-copy\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882626    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysconfig\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.884513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882647    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-cnibin\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882653    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-etc-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882647    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-iptables-alerter-script\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882699    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-conf-dir\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882700    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-etc-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882727    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-slash\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882750    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-conf\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882757    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882070    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-node-log\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882096    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882872    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-host\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882883    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-multus-socket-dir-parent\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.882974    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-cni-netd\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883026    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e362fef-5048-4204-92c4-c79c8206b72c-system-cni-dir\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883059    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysctl-d\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883088    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-sysconfig\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883124    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.885120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883132    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-var-lib-openvswitch\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883162    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1da582df-6d44-4b9a-a39f-a903d23c23b2-host-run-netns\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883173    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883217    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-systemd\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883260    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b54926db-c238-4f65-8827-207fd6181548-socket-dir\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883430    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-log-socket\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883468    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716a74a8-e82d-4a3b-8716-987ee521f598-host-run-ovn-kubernetes\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883503    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-host-slash\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883516    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716a74a8-e82d-4a3b-8716-987ee521f598-env-overrides\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.883882    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e362fef-5048-4204-92c4-c79c8206b72c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.884162    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-etc-tuned\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.884341    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716a74a8-e82d-4a3b-8716-987ee521f598-ovn-node-metrics-cert\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.885579    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4bc10e86-f575-4533-94a3-911036908965-agent-certs\") pod \"konnectivity-agent-9d52k\" (UID: \"4bc10e86-f575-4533-94a3-911036908965\") " pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:12.885791 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.885663    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-tmp\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.889719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.889685    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxx6\" (UniqueName: \"kubernetes.io/projected/1da582df-6d44-4b9a-a39f-a903d23c23b2-kube-api-access-snxx6\") pod \"multus-lh9v2\" (UID: \"1da582df-6d44-4b9a-a39f-a903d23c23b2\") " pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:12.893202 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.893177    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:12.893202 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.893204    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:12.893355 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.893215    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:12.893355 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.893281    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:13.393268298 +0000 UTC m=+3.036058443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:12.893355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.893309    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qb72\" (UniqueName: \"kubernetes.io/projected/b54926db-c238-4f65-8827-207fd6181548-kube-api-access-8qb72\") pod \"aws-ebs-csi-driver-node-bbrsp\" (UID: \"b54926db-c238-4f65-8827-207fd6181548\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:12.896646 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.896594    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2wg\" (UniqueName: \"kubernetes.io/projected/716a74a8-e82d-4a3b-8716-987ee521f598-kube-api-access-2q2wg\") pod \"ovnkube-node-86xfr\" (UID: \"716a74a8-e82d-4a3b-8716-987ee521f598\") " pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:12.897145 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.897123    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsj8\" (UniqueName: \"kubernetes.io/projected/a54c96c9-7cdd-4453-ae4e-2ee896c614a7-kube-api-access-gnsj8\") pod \"iptables-alerter-vg9pw\" (UID: \"a54c96c9-7cdd-4453-ae4e-2ee896c614a7\") " pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:12.897228 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.897180    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqwtq\" (UniqueName: \"kubernetes.io/projected/48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3-kube-api-access-kqwtq\") pod \"tuned-cbdwp\" (UID: \"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3\") " pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:12.897568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.897513    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs2l5\" (UniqueName: \"kubernetes.io/projected/6e362fef-5048-4204-92c4-c79c8206b72c-kube-api-access-rs2l5\") pod \"multus-additional-cni-plugins-5ndjs\" (UID: \"6e362fef-5048-4204-92c4-c79c8206b72c\") " pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:12.898833 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.898788    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmn99\" (UniqueName: \"kubernetes.io/projected/7dba23ae-c972-4567-9926-d61a07546e4d-kube-api-access-hmn99\") pod \"node-ca-jgkcv\" (UID: \"7dba23ae-c972-4567-9926-d61a07546e4d\") " pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:12.899025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.899005    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcn88\" (UniqueName: \"kubernetes.io/projected/96433f43-d834-4c3f-8d92-8cad28968754-kube-api-access-gcn88\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:12.899097 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.899078    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbq5s\" (UniqueName: \"kubernetes.io/projected/c64109e2-8108-4841-bac6-5117f7bf60a1-kube-api-access-rbq5s\") pod \"node-resolver-7c4sh\" (UID: \"c64109e2-8108-4841-bac6-5117f7bf60a1\") " pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:12.983871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.983644    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.984030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.983920    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-dbus\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.984030 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.983765    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:12.984030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.983967    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-kubelet-config\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.984162 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:12.984046    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:13.48402299 +0000 UTC m=+3.126813145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:12.984162 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.984062    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-kubelet-config\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:12.984258 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:12.984176    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3a36fe51-02f8-4227-876f-b081ce8df153-dbus\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:13.068521 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.068429    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:13.076277 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.076252    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:13.084958 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.084933    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp"
Sep 11 03:40:13.090525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.090505    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5ndjs"
Sep 11 03:40:13.095986 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.095963    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh9v2"
Sep 11 03:40:13.102353 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.102337    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jgkcv"
Sep 11 03:40:13.110651 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.110627    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vg9pw"
Sep 11 03:40:13.118113 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.118094    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7c4sh"
Sep 11 03:40:13.122542 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.122526    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cbdwp"
Sep 11 03:40:13.388720 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.388640    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:13.388864 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.388800    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:13.388909 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.388877    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:14.388860467 +0000 UTC m=+4.031650627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.489669    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.489733    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.489916    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.489956    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.489979    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.489995    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.489979    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:14.489961358 +0000 UTC m=+4.132751502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:13.490360 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.490057    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:14.49004246 +0000 UTC m=+4.132832599 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:13.632767 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:13.632744    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e362fef_5048_4204_92c4_c79c8206b72c.slice/crio-7ed4b929e207623a1cc2620d97599a7b4353fd5f95a50178030b271117b3a355 WatchSource:0}: Error finding container 7ed4b929e207623a1cc2620d97599a7b4353fd5f95a50178030b271117b3a355: Status 404 returned error can't find the container with id 7ed4b929e207623a1cc2620d97599a7b4353fd5f95a50178030b271117b3a355
Sep 11 03:40:13.653642 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:13.653619    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bb9617_98c1_4cfd_b0a0_ba5d1f0780d3.slice/crio-a46453bda63fb86daadd230cb7e053a9ec9cd026ea9914d5378045ceeeb63f4d WatchSource:0}: Error finding container a46453bda63fb86daadd230cb7e053a9ec9cd026ea9914d5378045ceeeb63f4d: Status 404 returned error can't find the container with id a46453bda63fb86daadd230cb7e053a9ec9cd026ea9914d5378045ceeeb63f4d
Sep 11 03:40:13.658377 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:13.658356    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dba23ae_c972_4567_9926_d61a07546e4d.slice/crio-064db0125728d113b500e5e1aa7aa2f606091044f6f9a9ef20bdbbd1b72c6751 WatchSource:0}: Error finding container 064db0125728d113b500e5e1aa7aa2f606091044f6f9a9ef20bdbbd1b72c6751: Status 404 returned error can't find the container with id 064db0125728d113b500e5e1aa7aa2f606091044f6f9a9ef20bdbbd1b72c6751
Sep 11 03:40:13.659463 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:13.659436    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb54926db_c238_4f65_8827_207fd6181548.slice/crio-3b943dbfdc2717cfab23d1e415e07be4d16b7b5e40c34c48f21cf6eeaedb8981 WatchSource:0}: Error finding container 3b943dbfdc2717cfab23d1e415e07be4d16b7b5e40c34c48f21cf6eeaedb8981: Status 404 returned error can't find the container with id 3b943dbfdc2717cfab23d1e415e07be4d16b7b5e40c34c48f21cf6eeaedb8981
Sep 11 03:40:13.660213 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:13.660189    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54c96c9_7cdd_4453_ae4e_2ee896c614a7.slice/crio-22d0589d25048462a95ad1eb8075ef9e2b2dac1372edc1a536d4554934aa9579 WatchSource:0}: Error finding container 22d0589d25048462a95ad1eb8075ef9e2b2dac1372edc1a536d4554934aa9579: Status 404 returned error can't find the container with id 22d0589d25048462a95ad1eb8075ef9e2b2dac1372edc1a536d4554934aa9579
Sep 11 03:40:13.811355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.811321    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 03:35:12 +0000 UTC" deadline="2028-05-11 08:22:55.733674769 +0000 UTC"
Sep 11 03:40:13.811355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.811353    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14596h42m41.922326597s"
Sep 11 03:40:13.866736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.866711    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:13.866881 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:13.866828    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:13.877931 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.877904    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"6af7035d1c208aec585db4ef593097d62bf4ca94336330f208a38d62768fd455"}
Sep 11 03:40:13.879012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.878982    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp" event={"ID":"b54926db-c238-4f65-8827-207fd6181548","Type":"ContainerStarted","Data":"3b943dbfdc2717cfab23d1e415e07be4d16b7b5e40c34c48f21cf6eeaedb8981"}
Sep 11 03:40:13.880100 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.880072    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgkcv" event={"ID":"7dba23ae-c972-4567-9926-d61a07546e4d","Type":"ContainerStarted","Data":"064db0125728d113b500e5e1aa7aa2f606091044f6f9a9ef20bdbbd1b72c6751"}
Sep 11 03:40:13.881054 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.881035    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cbdwp" event={"ID":"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3","Type":"ContainerStarted","Data":"a46453bda63fb86daadd230cb7e053a9ec9cd026ea9914d5378045ceeeb63f4d"}
Sep 11 03:40:13.882918 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.882898    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal" event={"ID":"85b0479989c71536ba20c64f751ffba4","Type":"ContainerStarted","Data":"46681798b45baa97bf2cc50a05e2c659aebfbd71710a858cf399eb41146784aa"}
Sep 11 03:40:13.885007 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.884987    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh9v2" event={"ID":"1da582df-6d44-4b9a-a39f-a903d23c23b2","Type":"ContainerStarted","Data":"626137b79b6d18481a70e3fb3b4e757dfc1affa4b385f29d25cc07ce458ba1c0"}
Sep 11 03:40:13.886146 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.886123    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7c4sh" event={"ID":"c64109e2-8108-4841-bac6-5117f7bf60a1","Type":"ContainerStarted","Data":"f64d0596abca4277aa40e7ee94929af1c2f8fdc888a3040272ce86d857acca07"}
Sep 11 03:40:13.887157 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.887132    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9d52k" event={"ID":"4bc10e86-f575-4533-94a3-911036908965","Type":"ContainerStarted","Data":"1ec4b1528e6f1013ec64232be6b8f87d4dcbf9e5c70abfe255b54b9dc2943a91"}
Sep 11 03:40:13.888190 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.888171    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vg9pw" event={"ID":"a54c96c9-7cdd-4453-ae4e-2ee896c614a7","Type":"ContainerStarted","Data":"22d0589d25048462a95ad1eb8075ef9e2b2dac1372edc1a536d4554934aa9579"}
Sep 11 03:40:13.889244 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.889224    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerStarted","Data":"7ed4b929e207623a1cc2620d97599a7b4353fd5f95a50178030b271117b3a355"}
Sep 11 03:40:13.899722 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:13.899685    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-129.ec2.internal" podStartSLOduration=1.899674316 podStartE2EDuration="1.899674316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:13.899505788 +0000 UTC m=+3.542295950" watchObservedRunningTime="2026-09-11 03:40:13.899674316 +0000 UTC m=+3.542464485"
Sep 11 03:40:14.395107 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.394455    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:14.395107 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.394625    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:14.395107 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.394688    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:16.394670922 +0000 UTC m=+6.037461077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:14.495068 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.495028    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:14.495253 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.495098    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:14.495253 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495246    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:14.495364 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495313    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:16.495293381 +0000 UTC m=+6.138083535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:14.495423 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495401    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:14.495423 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495415    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:14.495517 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495427    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:14.495517 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.495463    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:16.495452056 +0000 UTC m=+6.138242210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:14.812477 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.812332    2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 03:35:12 +0000 UTC" deadline="2028-05-07 23:38:55.1401812 +0000 UTC"
Sep 11 03:40:14.812477 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.812392    2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14515h58m40.327792996s"
Sep 11 03:40:14.868495 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.868010    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:14.868495 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.868137    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:14.868995 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.868836    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:14.868995 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:14.868953    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:14.900242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.900206    2553 generic.go:356] "Generic (PLEG): container finished" podID="f5493fdbc7f8db30fc5476d18f9a9b83" containerID="4c009f67482933000b464b41a0f5e0cb450a8498cbaef43de298a4ccba32f9ee" exitCode=0
Sep 11 03:40:14.901260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:14.901233    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal" event={"ID":"f5493fdbc7f8db30fc5476d18f9a9b83","Type":"ContainerDied","Data":"4c009f67482933000b464b41a0f5e0cb450a8498cbaef43de298a4ccba32f9ee"}
Sep 11 03:40:15.865964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:15.865872    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:15.866551 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:15.866040    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:15.934897 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:15.934735    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal" event={"ID":"f5493fdbc7f8db30fc5476d18f9a9b83","Type":"ContainerStarted","Data":"87806974e19e16a773c928c03f6df7d8c68ce266037a8e75361b17a7292f410a"}
Sep 11 03:40:16.408938 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:16.408276    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:16.408938 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.408464    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:16.408938 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.408542    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:20.408523813 +0000 UTC m=+10.051313967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:16.509106 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:16.509070    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:16.509267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:16.509135    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:16.509344 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509278    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:16.509390 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509342    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:20.509323998 +0000 UTC m=+10.152114159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:16.509783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509761    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:16.509896 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509791    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:16.509896 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509803    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:16.509896 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.509888    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:20.509871797 +0000 UTC m=+10.152661950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:16.867392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:16.866853    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:16.867392 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.866985    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:16.867392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:16.867022    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:16.867392 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:16.867142    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:17.866239 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:17.866208    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:17.866429 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:17.866335    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:18.866766 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:18.866724    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:18.867227 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:18.866724    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:18.867227 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:18.866895    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:18.867227 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:18.866990    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:19.866545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:19.866238    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:19.866711 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:19.866641    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:20.442639 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:20.442043    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:20.442639 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.442243    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:20.442639 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.442310    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:28.442290542 +0000 UTC m=+18.085080687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:20.542479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:20.542433    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:20.542654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:20.542499    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:20.542654 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542644    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:20.542783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542656    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:20.542783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542669    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:20.542783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542684    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:20.542783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542726    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:28.542704394 +0000 UTC m=+18.185494540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:20.542783 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.542745    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:28.542736376 +0000 UTC m=+18.185526517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:20.866918 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:20.866885    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:20.867107 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.867010    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:20.867406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:20.867234    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:20.867406 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:20.867365    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:21.866184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:21.866148    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:21.866629 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:21.866289    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:22.866377 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:22.866348    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:22.866838 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:22.866348    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:22.866838 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:22.866503    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:22.866838 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:22.866662    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:23.866497 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:23.866469    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:23.866845 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:23.866575    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:24.869672 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:24.869638    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:24.870094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:24.869685    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:24.870094 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:24.869797    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:24.870094 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:24.869884    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:25.866859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:25.866809    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:25.867050 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:25.866933    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:26.869802 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:26.869776    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:26.870267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:26.869778    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:26.870267 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:26.869901    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:26.870267 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:26.870015    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:27.865909 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:27.865879    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:27.866131 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:27.865996    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:28.501185 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:28.501138    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:28.501630 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.501322    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:28.501630 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.501406    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:44.50138588 +0000 UTC m=+34.144176035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:28.602212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:28.602178    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:28.602387 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:28.602240    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:28.602387 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602352    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:28.602387 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602367    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:28.602533 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602394    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:28.602533 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602405    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:28.602533 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602409    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:44.602396384 +0000 UTC m=+34.245186537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:28.602533 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.602445    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:40:44.602435034 +0000 UTC m=+34.245225173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:28.866389 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:28.866354    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:28.866576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:28.866364    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:28.866576 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.866461    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:28.866680 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:28.866572    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:29.866770 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:29.866738    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:29.867190 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:29.866860    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:30.867270 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:30.867239    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:30.867700 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:30.867339    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:30.867700 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:30.867452    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:30.867700 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:30.867578    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:31.866723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.866697    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:31.866858 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:31.866835    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:31.968296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.968073    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"4c57fdc017b3304aca5294e260649499a7899c0a2116d6537dcd113355fdd459"}
Sep 11 03:40:31.969738 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.969722    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp" event={"ID":"b54926db-c238-4f65-8827-207fd6181548","Type":"ContainerStarted","Data":"4cac33f78d2653fb107e2b6bd180a850e28e52be6c00e6f5974a9a418b1b76d9"}
Sep 11 03:40:31.971261 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.971226    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jgkcv" event={"ID":"7dba23ae-c972-4567-9926-d61a07546e4d","Type":"ContainerStarted","Data":"e278c42c669e31203e06000d7d909e467f530988f560f868f934f15c62d567bc"}
Sep 11 03:40:31.973097 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.973070    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cbdwp" event={"ID":"48bb9617-98c1-4cfd-b0a0-ba5d1f0780d3","Type":"ContainerStarted","Data":"f71ac8657448607ab7e396e1600df17cf3d8df6587064efcff7f6ae740c50dad"}
Sep 11 03:40:31.974721 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.974695    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh9v2" event={"ID":"1da582df-6d44-4b9a-a39f-a903d23c23b2","Type":"ContainerStarted","Data":"c4a5ba1d74990f754461dfe86f1d4fcdfda0ef1574eae6d58df02d6e1266001f"}
Sep 11 03:40:31.976254 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.976228    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7c4sh" event={"ID":"c64109e2-8108-4841-bac6-5117f7bf60a1","Type":"ContainerStarted","Data":"3da84b9a6e9f31471b3a5eccb07b8cf4110316d1745223a5df25a004d00776c3"}
Sep 11 03:40:31.977767 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.977742    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9d52k" event={"ID":"4bc10e86-f575-4533-94a3-911036908965","Type":"ContainerStarted","Data":"537ebc1422d6f2fa25c46cce33a2d1d12b0898ed946d406575cbb0b6d8fe8027"}
Sep 11 03:40:31.979312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.979290    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="2091a3c40c53b7d0168900ed472dd8907e08238a61eb4e20f990783fda11b5a7" exitCode=0
Sep 11 03:40:31.979384 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.979325    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"2091a3c40c53b7d0168900ed472dd8907e08238a61eb4e20f990783fda11b5a7"}
Sep 11 03:40:31.985789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.985738    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jgkcv" podStartSLOduration=3.332745004 podStartE2EDuration="20.985720778s" totalImagesPullingTime="17.652975774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:31.985277161 +0000 UTC m=+21.628067322" watchObservedRunningTime="2026-09-11 03:40:31.985720778 +0000 UTC m=+21.628510942"
Sep 11 03:40:31.985914 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.985862    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-129.ec2.internal" podStartSLOduration=19.985852994 podStartE2EDuration="19.985852994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:15.952791859 +0000 UTC m=+5.595582023" watchObservedRunningTime="2026-09-11 03:40:31.985852994 +0000 UTC m=+21.628643160"
Sep 11 03:40:31.998789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:31.998712    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9d52k" podStartSLOduration=11.776108002 podStartE2EDuration="21.998694995s" totalImagesPullingTime="10.222586993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:31.998649555 +0000 UTC m=+21.641439731" watchObservedRunningTime="2026-09-11 03:40:31.998694995 +0000 UTC m=+21.641485158"
Sep 11 03:40:32.041597 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.041554    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lh9v2" podStartSLOduration=4.340242689 podStartE2EDuration="22.041538088s" totalImagesPullingTime="17.701295399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:32.020526775 +0000 UTC m=+21.663316939" watchObservedRunningTime="2026-09-11 03:40:32.041538088 +0000 UTC m=+21.684328248"
Sep 11 03:40:32.055356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.055317    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7c4sh" podStartSLOduration=3.392228951 podStartE2EDuration="21.055303323s" totalImagesPullingTime="17.663074372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:32.055025019 +0000 UTC m=+21.697815182" watchObservedRunningTime="2026-09-11 03:40:32.055303323 +0000 UTC m=+21.698093479"
Sep 11 03:40:32.073697 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.073655    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cbdwp" podStartSLOduration=3.399573651 podStartE2EDuration="21.073639426s" totalImagesPullingTime="17.674065775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:32.073301113 +0000 UTC m=+21.716091296" watchObservedRunningTime="2026-09-11 03:40:32.073639426 +0000 UTC m=+21.716429587"
Sep 11 03:40:32.587656 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.587630    2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 03:40:32.844442 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.844347    2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T03:40:32.587652025Z","UUID":"2fc0ae6d-da80-490d-bd30-33f4a8e86e32","Handler":null,"Name":"","Endpoint":""}
Sep 11 03:40:32.846029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.846004    2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 03:40:32.846159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.846045    2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 03:40:32.866163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.866139    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:32.866287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.866139    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:32.866287 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:32.866247    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:32.866368 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:32.866321    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:32.983283 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.983245    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vg9pw" event={"ID":"a54c96c9-7cdd-4453-ae4e-2ee896c614a7","Type":"ContainerStarted","Data":"8b8a04241cd2e1877eb35439ab1dddb9be653ffd264e0b32433c684a40bdde34"}
Sep 11 03:40:32.986127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.986101    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"c20590f38f572e3df7cc2c78b66b8d81b66544cfc4830ad9794a5ad90ab8e924"}
Sep 11 03:40:32.986256 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.986176    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"855f4aba65fd864da8d7387b765e9dfa49695917b3248de98c424cd37583ab9b"}
Sep 11 03:40:32.986256 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.986221    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"d409cabceee4ced98ecc1e1b44d64a602deaf2e0907884c508efb06c8deb791f"}
Sep 11 03:40:32.986370 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.986262    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"870976b13988e87b27ff182c2fcec06a9cfc52bb0fe70968a4400ba4072a9c89"}
Sep 11 03:40:32.986370 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.986290    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"cc602b58610863ceb6d99b77c8343cc47ff0429e7b4f7c6179a226e4511c0d00"}
Sep 11 03:40:32.988654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:32.988571    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp" event={"ID":"b54926db-c238-4f65-8827-207fd6181548","Type":"ContainerStarted","Data":"a8384c80caefb81a728d4a66cf83e42810aa481853ef31e4a36f8a8bfc0dbdd4"}
Sep 11 03:40:33.866560 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:33.866528    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:33.866741 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:33.866649    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:33.993855 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:33.993755    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp" event={"ID":"b54926db-c238-4f65-8827-207fd6181548","Type":"ContainerStarted","Data":"8519d0c287d42a83e96bd5d04b5e697567e9ca45abcb370486e01156ec9c9d2e"}
Sep 11 03:40:34.012945 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:34.012897    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vg9pw" podStartSLOduration=5.346220756 podStartE2EDuration="23.012881027s" totalImagesPullingTime="17.666660271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:32.997866104 +0000 UTC m=+22.640656266" watchObservedRunningTime="2026-09-11 03:40:34.012881027 +0000 UTC m=+23.655671186"
Sep 11 03:40:34.013221 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:34.013197    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bbrsp" podStartSLOduration=3.385449683 podStartE2EDuration="23.013190181s" totalImagesPullingTime="19.627740498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:34.011729942 +0000 UTC m=+23.654520106" watchObservedRunningTime="2026-09-11 03:40:34.013190181 +0000 UTC m=+23.655980342"
Sep 11 03:40:34.866691 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:34.866660    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:34.866923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:34.866670    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:34.866923 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:34.866796    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:34.866923 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:34.866891    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:34.998997 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:34.998961    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"e4b8514bce90a2bcf63f8daa8c7da8d58ac4a29a17d5ffc649c74354d396b602"}
Sep 11 03:40:35.865775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:35.865742    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:35.865966 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:35.865874    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:36.568091 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:36.567954    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:36.568479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:36.568454    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:36.869334 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:36.869266    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:36.869458 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:36.869271    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:36.869458 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:36.869359    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:36.869458 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:36.869442    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:37.004715 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:37.004680    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="29549c54347d4380214ba22588dfc3c83fa6295fd803c099a91f5cb7224baf95" exitCode=0
Sep 11 03:40:37.004871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:37.004740    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"29549c54347d4380214ba22588dfc3c83fa6295fd803c099a91f5cb7224baf95"}
Sep 11 03:40:37.008605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:37.008571    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" event={"ID":"716a74a8-e82d-4a3b-8716-987ee521f598","Type":"ContainerStarted","Data":"fd9cbfd147bc2e2dd82ddbe8c298d08058c84f4f853afc69bca4c1f428199ad6"}
Sep 11 03:40:37.049023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:37.048979    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr" podStartSLOduration=8.897158293 podStartE2EDuration="27.048965451s" totalImagesPullingTime="18.151807158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:37.047306167 +0000 UTC m=+26.690096329" watchObservedRunningTime="2026-09-11 03:40:37.048965451 +0000 UTC m=+26.691755613"
Sep 11 03:40:37.866534 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:37.866484    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:37.867483 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:37.866626    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:38.013273 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.013246    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="bb59ab128978374873037f9d9a3ab1e4fa7ac60626dcecd184fca9943b361d1c" exitCode=0
Sep 11 03:40:38.013427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.013329    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"bb59ab128978374873037f9d9a3ab1e4fa7ac60626dcecd184fca9943b361d1c"}
Sep 11 03:40:38.013751 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.013728    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:38.013861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.013806    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:38.013900 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.013873    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:38.030666 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.030630    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:38.032933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.032912    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:40:38.487619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.487320    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wqbk"]
Sep 11 03:40:38.487782 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.487642    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:38.487782 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:38.487765    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:38.490572 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.490546    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdssj"]
Sep 11 03:40:38.490693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.490669    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:38.490802 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:38.490773    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:38.491195 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.490971    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s5gzw"]
Sep 11 03:40:38.491195 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:38.491070    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:38.491195 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:38.491162    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:39.017789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.017758    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="1a0e37cc850b8e31d06a7a2ebfdc842e839790a335a97290e7a41eb008c32b7a" exitCode=0
Sep 11 03:40:39.018181 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.017851    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"1a0e37cc850b8e31d06a7a2ebfdc842e839790a335a97290e7a41eb008c32b7a"}
Sep 11 03:40:39.126128 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.126098    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:39.126313 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.126210    2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 03:40:39.126668 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.126646    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9d52k"
Sep 11 03:40:39.866237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.866207    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:39.866237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.866221    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:39.866450 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:39.866207    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:39.866450 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:39.866325    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:39.866450 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:39.866388    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:39.866598 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:39.866514    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:41.866423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:41.866388    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:41.866900 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:41.866388    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:41.866900 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:41.866526    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:41.866900 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:41.866588    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:41.866900 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:41.866388    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:41.866900 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:41.866694    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:43.866676 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:43.866643    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:43.867105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:43.866679    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:43.867105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:43.866643    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:43.867105 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:43.866765    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wqbk" podUID="96433f43-d834-4c3f-8d92-8cad28968754"
Sep 11 03:40:43.867105 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:43.866826    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-s5gzw" podUID="3a36fe51-02f8-4227-876f-b081ce8df153"
Sep 11 03:40:43.867105 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:43.866903    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdssj" podUID="519f0793-010d-4255-89e3-c1c1c73574de"
Sep 11 03:40:44.518093 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.518060    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:44.518236 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.518197    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:44.518276 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.518254    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.518240066 +0000 UTC m=+66.161030205 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 03:40:44.613617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.613591    2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-129.ec2.internal" event="NodeReady"
Sep 11 03:40:44.613754 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.613736    2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 03:40:44.618855 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.618835    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:44.618911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.618874    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:44.618998 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.618985    2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:44.619054 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.619037    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret podName:3a36fe51-02f8-4227-876f-b081ce8df153 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.619024135 +0000 UTC m=+66.261814288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret") pod "global-pull-secret-syncer-s5gzw" (UID: "3a36fe51-02f8-4227-876f-b081ce8df153") : object "kube-system"/"original-pull-secret" not registered
Sep 11 03:40:44.619054 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.618988    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 03:40:44.619121 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.619062    2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 03:40:44.619121 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.619075    2553 projected.go:196] Error preparing data for projected volume kube-api-access-qffmg for pod openshift-network-diagnostics/network-check-target-pdssj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:44.619121 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.619104    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg podName:519f0793-010d-4255-89e3-c1c1c73574de nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.619097751 +0000 UTC m=+66.261887890 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qffmg" (UniqueName: "kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg") pod "network-check-target-pdssj" (UID: "519f0793-010d-4255-89e3-c1c1c73574de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 03:40:44.646077 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.646026    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:40:44.671114 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.671089    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pfwn5"]
Sep 11 03:40:44.671272 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.671210    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.674095 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.674001    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 03:40:44.674095 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.674063    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 03:40:44.674286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.674225    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2llnq\""
Sep 11 03:40:44.674286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.674235    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 03:40:44.680071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.680050    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 03:40:44.685871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.685849    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:40:44.685955 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.685887    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wjcjz"]
Sep 11 03:40:44.685955 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.685904    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.688397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.688379    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 03:40:44.688486 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.688476    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 03:40:44.688576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.688561    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qlqn4\""
Sep 11 03:40:44.707987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.707968    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pfwn5"]
Sep 11 03:40:44.708080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.708007    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjcjz"]
Sep 11 03:40:44.708121 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.708090    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.710888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.710840    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 03:40:44.710888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.710866    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 03:40:44.710888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.710875    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qd2rt\""
Sep 11 03:40:44.711081 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.710931    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 03:40:44.797869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.797836    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:44.819755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819731    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.819865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819756    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66hp\" (UniqueName: \"kubernetes.io/projected/2396b115-42d4-482c-849c-cddb2ea07ba5-kube-api-access-b66hp\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.819865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819776    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.819865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819800    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.819865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819857    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819875    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2wm2\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819895    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819955    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.819996    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/344210ec-7df8-4263-a6ca-3fda63110b03-tmp-dir\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.820148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.820019    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrcd\" (UniqueName: \"kubernetes.io/projected/344210ec-7df8-4263-a6ca-3fda63110b03-kube-api-access-xxrcd\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.820148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.820035    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.820049    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2396b115-42d4-482c-849c-cddb2ea07ba5-cert\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.820148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.820120    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.820148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.820142    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/344210ec-7df8-4263-a6ca-3fda63110b03-config-volume\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.882379 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.882353    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:44.921364 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921337    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2wm2\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921373    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921440    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921462    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/344210ec-7df8-4263-a6ca-3fda63110b03-tmp-dir\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.921571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921479    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrcd\" (UniqueName: \"kubernetes.io/projected/344210ec-7df8-4263-a6ca-3fda63110b03-kube-api-access-xxrcd\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.921571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921496    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921520    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2396b115-42d4-482c-849c-cddb2ea07ba5-cert\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.921571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921553    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921577    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/344210ec-7df8-4263-a6ca-3fda63110b03-config-volume\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921621    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921643    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66hp\" (UniqueName: \"kubernetes.io/projected/2396b115-42d4-482c-849c-cddb2ea07ba5-kube-api-access-b66hp\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921668    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921689    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.921755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921714    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.922059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921899    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/344210ec-7df8-4263-a6ca-3fda63110b03-tmp-dir\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.922059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.921952    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.922206 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.922183    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:40:44.922262 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.922208    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:40:44.923096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.922713    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/344210ec-7df8-4263-a6ca-3fda63110b03-config-volume\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.923096 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.922966    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:45.422250527 +0000 UTC m=+35.065040668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:40:44.923096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.922985    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.923096 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.922968    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:40:44.923332 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:44.923118    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:45.423094516 +0000 UTC m=+35.065884666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:40:44.923618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.923595    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.927503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.927366    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.927503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.927406    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2396b115-42d4-482c-849c-cddb2ea07ba5-cert\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:44.927503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.927457    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.930159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.930121    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2wm2\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.930305 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.930284    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrcd\" (UniqueName: \"kubernetes.io/projected/344210ec-7df8-4263-a6ca-3fda63110b03-kube-api-access-xxrcd\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:44.930461 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.930446    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:44.931528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:44.931507    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66hp\" (UniqueName: \"kubernetes.io/projected/2396b115-42d4-482c-849c-cddb2ea07ba5-kube-api-access-b66hp\") pod \"ingress-canary-wjcjz\" (UID: \"2396b115-42d4-482c-849c-cddb2ea07ba5\") " pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:45.016170 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.016146    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wjcjz"
Sep 11 03:40:45.036896 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.036865    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="3c663d081443d08d48b29421a949746aad89e8adb91f734c7402dc699b96b8da" exitCode=0
Sep 11 03:40:45.036987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.036924    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"3c663d081443d08d48b29421a949746aad89e8adb91f734c7402dc699b96b8da"}
Sep 11 03:40:45.347196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.347121    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wjcjz"]
Sep 11 03:40:45.350156 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:40:45.350130    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2396b115_42d4_482c_849c_cddb2ea07ba5.slice/crio-fed3558e2069d25e27529af38cb5f8b261e7e4d9db1a761fc9c084529887c30e WatchSource:0}: Error finding container fed3558e2069d25e27529af38cb5f8b261e7e4d9db1a761fc9c084529887c30e: Status 404 returned error can't find the container with id fed3558e2069d25e27529af38cb5f8b261e7e4d9db1a761fc9c084529887c30e
Sep 11 03:40:45.425459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.425428    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:45.425459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.425462    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:45.425662 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:45.425581    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:40:45.425662 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:45.425587    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:40:45.425662 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:45.425605    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:40:45.425662 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:45.425633    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:46.425618354 +0000 UTC m=+36.068408494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:40:45.425662 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:45.425651    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:46.425638571 +0000 UTC m=+36.068428741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:40:45.765734 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.765702    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:45.866390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.866357    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:40:45.866390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.866377    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:40:45.866595 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.866534    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:40:45.869355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.869340    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 03:40:45.870644 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.870629    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 03:40:45.870990 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.870974    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 03:40:45.871046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.871025    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hqw27\""
Sep 11 03:40:45.871129 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.871053    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 03:40:45.871183 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:45.871131    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sfchv\""
Sep 11 03:40:46.045503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.045420    2553 generic.go:356] "Generic (PLEG): container finished" podID="6e362fef-5048-4204-92c4-c79c8206b72c" containerID="ed07de889060b9ea960b0318ac6026d0e48249eae7cbddd8a11047a42bee0c3b" exitCode=0
Sep 11 03:40:46.046442 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.045490    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerDied","Data":"ed07de889060b9ea960b0318ac6026d0e48249eae7cbddd8a11047a42bee0c3b"}
Sep 11 03:40:46.046960 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.046932    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjcjz" event={"ID":"2396b115-42d4-482c-849c-cddb2ea07ba5","Type":"ContainerStarted","Data":"fed3558e2069d25e27529af38cb5f8b261e7e4d9db1a761fc9c084529887c30e"}
Sep 11 03:40:46.175154 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.174969    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:46.266581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.266549    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:46.363942 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.363853    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:46.433025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.432988    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:46.433025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.433034    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:46.433261 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:46.433154    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:40:46.433261 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:46.433170    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:40:46.433261 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:46.433170    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:40:46.433261 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:46.433232    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:48.433218294 +0000 UTC m=+38.076008434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:40:46.433261 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:46.433250    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:48.433241267 +0000 UTC m=+38.076031406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:40:46.828614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.828584    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:46.929408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:46.929370    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:47.052941 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.052908    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" event={"ID":"6e362fef-5048-4204-92c4-c79c8206b72c","Type":"ContainerStarted","Data":"e759e652f2328f85390242876fe7d010945c5ccfb83431dbcebb371aa99ebf29"}
Sep 11 03:40:47.078316 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.078263    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5ndjs" podStartSLOduration=8.476883459 podStartE2EDuration="37.078246368s" totalImagesPullingTime="28.601362909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:47.077672522 +0000 UTC m=+36.720462684" watchObservedRunningTime="2026-09-11 03:40:47.078246368 +0000 UTC m=+36.721036530"
Sep 11 03:40:47.229898 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.229809    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:47.284995 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.284951    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:47.345456 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.345409    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:47.554561 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.554525    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:47.613558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:47.613527    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:48.359016 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.358977    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:48.415094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.415050    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:48.449991 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.449955    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:48.449991 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.449991    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:48.450165 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:48.450100    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:40:48.450165 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:48.450105    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:40:48.450165 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:48.450122    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:40:48.450165 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:48.450161    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:52.450138132 +0000 UTC m=+42.092928271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:40:48.450288 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:48.450174    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:40:52.450168594 +0000 UTC m=+42.092958733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:40:48.753883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.753768    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:48.855706 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:48.855671    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:49.059453 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:49.059413    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wjcjz" event={"ID":"2396b115-42d4-482c-849c-cddb2ea07ba5","Type":"ContainerStarted","Data":"1c7a3f35611f81d231bb410630188073f24ef29be906ef7c7ade473d745bcba0"}
Sep 11 03:40:49.076247 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:49.076190    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wjcjz" podStartSLOduration=2.447556391 podStartE2EDuration="5.076171804s" totalImagesPullingTime="2.628615413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:40:49.076125826 +0000 UTC m=+38.718916002" watchObservedRunningTime="2026-09-11 03:40:49.076171804 +0000 UTC m=+38.718961968"
Sep 11 03:40:49.664222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:49.664181    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:49.764790 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:49.764755    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:50.148204 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:50.148153    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:51.530434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:51.530403    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:51.596189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:51.596144    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:52.477514 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:52.477473    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:40:52.477514 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:52.477515    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:40:52.477764 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:52.477630    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:40:52.477764 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:52.477688    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:00.477671189 +0000 UTC m=+50.120461347 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:40:52.477764 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:52.477743    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:40:52.477764 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:52.477764    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:40:52.477996 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:40:52.477832    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:00.477802218 +0000 UTC m=+50.120592360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:40:58.219082 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:58.219036    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:40:58.282232 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:40:58.282196    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:00.533057 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:00.533014    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:00.533057 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:00.533055    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:00.533569 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:00.533149    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:41:00.533569 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:00.533157    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:41:00.533569 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:00.533175    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:41:00.533569 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:00.533214    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.533200964 +0000 UTC m=+66.175991104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:41:00.533569 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:00.533247    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.533233561 +0000 UTC m=+66.176023725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:41:01.277329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:01.277296    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:01.378376 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:01.378337    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:03.463522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:03.463489    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:03.517451 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:03.517417    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:04.688149 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.688117    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-csgkk"]
Sep 11 03:41:04.690005 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.689986    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.692971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.692953    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fjp5w\""
Sep 11 03:41:04.693456 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.693433    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 03:41:04.694260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.694243    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 03:41:04.695038 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.694959    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 03:41:04.695260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.695239    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 03:41:04.702966 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.702945    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 03:41:04.706605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.706585    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-csgkk"]
Sep 11 03:41:04.763396 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.763359    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d63a06-3620-4422-aecd-45eb88efe8e9-serving-cert\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.763576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.763415    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvcwf\" (UniqueName: \"kubernetes.io/projected/44d63a06-3620-4422-aecd-45eb88efe8e9-kube-api-access-zvcwf\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.763576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.763506    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-trusted-ca\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.763681 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.763585    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-config\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.782101 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.782060    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"]
Sep 11 03:41:04.783911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.783896    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"]
Sep 11 03:41:04.784030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.784013    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.785311 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.785293    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.787862 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.787839    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z556l"]
Sep 11 03:41:04.788103 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.788087    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 03:41:04.788170 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.788118    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 03:41:04.788344 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.788329    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 03:41:04.788419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.788378    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 03:41:04.788614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.788598    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zr76h\""
Sep 11 03:41:04.789947 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.789931    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 03:41:04.790026 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.789950    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mfkns\""
Sep 11 03:41:04.790295 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.790278    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l"
Sep 11 03:41:04.790707 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.790680    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 03:41:04.792770 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.792751    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t5zm8\""
Sep 11 03:41:04.798356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.798338    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"]
Sep 11 03:41:04.798978 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.798962    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"]
Sep 11 03:41:04.801209 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.801189    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z556l"]
Sep 11 03:41:04.864832 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.864785    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-config\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.865001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.864843    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.865001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.864886    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th55x\" (UniqueName: \"kubernetes.io/projected/b08a89f9-0d79-4493-8bdc-314fcacf7f4b-kube-api-access-th55x\") pod \"network-check-source-6f8b768455-z556l\" (UID: \"b08a89f9-0d79-4493-8bdc-314fcacf7f4b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l"
Sep 11 03:41:04.865001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.864909    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ddbeddec-7069-47bb-b860-7ecfade68cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.865001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.864942    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d63a06-3620-4422-aecd-45eb88efe8e9-serving-cert\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.865333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865023    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvcwf\" (UniqueName: \"kubernetes.io/projected/44d63a06-3620-4422-aecd-45eb88efe8e9-kube-api-access-zvcwf\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.865333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865062    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.865333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865081    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv7qz\" (UniqueName: \"kubernetes.io/projected/ddbeddec-7069-47bb-b860-7ecfade68cda-kube-api-access-rv7qz\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.865333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865101    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5cc12f0f-c45e-44f2-a024-97e95eebd129-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.865333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865176    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-trusted-ca\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.865618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865602    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-config\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.865862 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.865843    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d63a06-3620-4422-aecd-45eb88efe8e9-trusted-ca\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.868289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.868270    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d63a06-3620-4422-aecd-45eb88efe8e9-serving-cert\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.874376 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.874351    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvcwf\" (UniqueName: \"kubernetes.io/projected/44d63a06-3620-4422-aecd-45eb88efe8e9-kube-api-access-zvcwf\") pod \"console-operator-696884fb9c-csgkk\" (UID: \"44d63a06-3620-4422-aecd-45eb88efe8e9\") " pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:04.917746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.917706    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:04.965769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965694    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.965769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965728    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv7qz\" (UniqueName: \"kubernetes.io/projected/ddbeddec-7069-47bb-b860-7ecfade68cda-kube-api-access-rv7qz\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.965769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965745    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5cc12f0f-c45e-44f2-a024-97e95eebd129-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.966022 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965797    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.966022 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965842    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th55x\" (UniqueName: \"kubernetes.io/projected/b08a89f9-0d79-4493-8bdc-314fcacf7f4b-kube-api-access-th55x\") pod \"network-check-source-6f8b768455-z556l\" (UID: \"b08a89f9-0d79-4493-8bdc-314fcacf7f4b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l"
Sep 11 03:41:04.966022 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:04.965881    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:04.966022 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:04.965951    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:05.465933305 +0000 UTC m=+55.108723486 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:04.966022 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.965891    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ddbeddec-7069-47bb-b860-7ecfade68cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.966273 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:04.965962    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:04.966273 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:04.966091    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:05.466077903 +0000 UTC m=+55.108868046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:04.966553 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.966532    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5cc12f0f-c45e-44f2-a024-97e95eebd129-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:04.966614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.966553    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ddbeddec-7069-47bb-b860-7ecfade68cda-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:04.974380 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.974352    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th55x\" (UniqueName: \"kubernetes.io/projected/b08a89f9-0d79-4493-8bdc-314fcacf7f4b-kube-api-access-th55x\") pod \"network-check-source-6f8b768455-z556l\" (UID: \"b08a89f9-0d79-4493-8bdc-314fcacf7f4b\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l"
Sep 11 03:41:04.975617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:04.975600    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv7qz\" (UniqueName: \"kubernetes.io/projected/ddbeddec-7069-47bb-b860-7ecfade68cda-kube-api-access-rv7qz\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:05.001743 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.001716    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:05.017841 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.017795    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:05.106484 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.106441    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l"
Sep 11 03:41:05.270638 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.270607    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-csgkk"]
Sep 11 03:41:05.273068 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:05.273044    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d63a06_3620_4422_aecd_45eb88efe8e9.slice/crio-479ca4d84f890cea181cc4163cc6ebfe4f861c9633cfdb2729f85f0e44b42134 WatchSource:0}: Error finding container 479ca4d84f890cea181cc4163cc6ebfe4f861c9633cfdb2729f85f0e44b42134: Status 404 returned error can't find the container with id 479ca4d84f890cea181cc4163cc6ebfe4f861c9633cfdb2729f85f0e44b42134
Sep 11 03:41:05.373930 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.373898    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z556l"]
Sep 11 03:41:05.377255 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:05.377228    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08a89f9_0d79_4493_8bdc_314fcacf7f4b.slice/crio-9517893e44f978c7c8e001f800b24d5cd1237e1fd3185b6a0e67c93110672917 WatchSource:0}: Error finding container 9517893e44f978c7c8e001f800b24d5cd1237e1fd3185b6a0e67c93110672917: Status 404 returned error can't find the container with id 9517893e44f978c7c8e001f800b24d5cd1237e1fd3185b6a0e67c93110672917
Sep 11 03:41:05.470444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.470401    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:05.470620 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:05.470458    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:05.470620 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:05.470524    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:06.470506941 +0000 UTC m=+56.113297084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:05.470620 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:05.470551    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:05.470749 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:05.470684    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:05.470749 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:05.470729    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:06.470715795 +0000 UTC m=+56.113505937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:06.101189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:06.101141    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l" event={"ID":"b08a89f9-0d79-4493-8bdc-314fcacf7f4b","Type":"ContainerStarted","Data":"9517893e44f978c7c8e001f800b24d5cd1237e1fd3185b6a0e67c93110672917"}
Sep 11 03:41:06.102702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:06.102661    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" event={"ID":"44d63a06-3620-4422-aecd-45eb88efe8e9","Type":"ContainerStarted","Data":"479ca4d84f890cea181cc4163cc6ebfe4f861c9633cfdb2729f85f0e44b42134"}
Sep 11 03:41:06.484287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:06.484011    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:06.484287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:06.484139    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:06.484287 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:06.484158    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:06.484287 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:06.484271    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:08.484246917 +0000 UTC m=+58.127037057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:06.484613 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:06.484292    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:06.484613 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:06.484388    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:08.484370193 +0000 UTC m=+58.127160348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:08.282797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.282760    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:08.336470 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.336423    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:08.503040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.503000    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:08.503226 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.503113    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:08.503226 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:08.503137    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:08.503226 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:08.503220    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:12.503199971 +0000 UTC m=+62.145990113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:08.503351 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:08.503236    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:08.503351 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:08.503290    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:12.503272696 +0000 UTC m=+62.146062840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:08.842025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.841990    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9qbbh"]
Sep 11 03:41:08.843883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.843866    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:08.847034 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.847013    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 03:41:08.848302 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.848283    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8h9gh\""
Sep 11 03:41:08.848302 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.848291    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 03:41:08.848453 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.848300    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 03:41:08.848453 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.848301    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 03:41:08.865637 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.865612    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9qbbh"]
Sep 11 03:41:08.877511 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.877485    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:08.941546 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:08.941445    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.007135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.007098    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-crio-socket\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.007310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.007149    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.007310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.007245    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w86mc\" (UniqueName: \"kubernetes.io/projected/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-api-access-w86mc\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.007397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.007308    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-data-volume\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.007397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.007350    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.094177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.094138    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.107762 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107734    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-crio-socket\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.107892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107774    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.107892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107799    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86mc\" (UniqueName: \"kubernetes.io/projected/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-api-access-w86mc\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.107892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107853    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-data-volume\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.107892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107882    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.108080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.107891    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-crio-socket\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.108080 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.107920    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:09.108080 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.107976    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 03:41:09.108080 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.107983    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:09.607968695 +0000 UTC m=+59.250758870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:09.108080 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.108033    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:09.60801852 +0000 UTC m=+59.250808689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : configmap "kube-rbac-proxy" not found
Sep 11 03:41:09.108310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.108288    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-data-volume\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.111705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.111680    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l" event={"ID":"b08a89f9-0d79-4493-8bdc-314fcacf7f4b","Type":"ContainerStarted","Data":"6a60c98062044a85c5624c47ee7f577afe4d564f83a66a38baa5926b68a02aa4"}
Sep 11 03:41:09.113450 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.113430    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/0.log"
Sep 11 03:41:09.113558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.113472    2553 generic.go:356] "Generic (PLEG): container finished" podID="44d63a06-3620-4422-aecd-45eb88efe8e9" containerID="811f57f8ee932fe5a04b8bcd5e02775c29dbc3ed132b25248cb0d4589c071275" exitCode=255
Sep 11 03:41:09.113558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.113519    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" event={"ID":"44d63a06-3620-4422-aecd-45eb88efe8e9","Type":"ContainerDied","Data":"811f57f8ee932fe5a04b8bcd5e02775c29dbc3ed132b25248cb0d4589c071275"}
Sep 11 03:41:09.113782 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.113761    2553 scope.go:117] "RemoveContainer" containerID="811f57f8ee932fe5a04b8bcd5e02775c29dbc3ed132b25248cb0d4589c071275"
Sep 11 03:41:09.119664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.119643    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w86mc\" (UniqueName: \"kubernetes.io/projected/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-api-access-w86mc\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.128049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.128003    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z556l" podStartSLOduration=1.825751065 podStartE2EDuration="5.127986613s" totalImagesPullingTime="3.302235548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:09.126719875 +0000 UTC m=+58.769510061" watchObservedRunningTime="2026-09-11 03:41:09.127986613 +0000 UTC m=+58.770776777"
Sep 11 03:41:09.155621 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.155589    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.280411 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.280374    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.331431 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.331399    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.611993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.611950    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.612216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.612026    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:09.612216 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.612109    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 03:41:09.612216 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.612116    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:09.612341 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.612201    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:10.61217971 +0000 UTC m=+60.254969864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : configmap "kube-rbac-proxy" not found
Sep 11 03:41:09.612341 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:09.612266    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:10.612249866 +0000 UTC m=+60.255040019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:09.756438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.756399    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:09.860173 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:09.860129    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:10.031004 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.030978    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-86xfr"
Sep 11 03:41:10.117831 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.117801    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/1.log"
Sep 11 03:41:10.118222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.118207    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/0.log"
Sep 11 03:41:10.118288 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.118240    2553 generic.go:356] "Generic (PLEG): container finished" podID="44d63a06-3620-4422-aecd-45eb88efe8e9" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c" exitCode=255
Sep 11 03:41:10.118348 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.118324    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" event={"ID":"44d63a06-3620-4422-aecd-45eb88efe8e9","Type":"ContainerDied","Data":"673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"}
Sep 11 03:41:10.118418 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.118385    2553 scope.go:117] "RemoveContainer" containerID="811f57f8ee932fe5a04b8bcd5e02775c29dbc3ed132b25248cb0d4589c071275"
Sep 11 03:41:10.118621 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.118605    2553 scope.go:117] "RemoveContainer" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"
Sep 11 03:41:10.118792 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:10.118772    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:10.180414 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.180379    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:10.239366 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.239330    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:10.620825 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.620775    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:10.621164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.620874    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:10.621164 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:10.620933    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:10.621164 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:10.620994    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 03:41:10.621164 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:10.621018    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:12.62100025 +0000 UTC m=+62.263790407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:10.621164 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:10.621047    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:12.621033678 +0000 UTC m=+62.263823823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : configmap "kube-rbac-proxy" not found
Sep 11 03:41:10.691272 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.691242    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7c4sh_c64109e2-8108-4841-bac6-5117f7bf60a1/dns-node-resolver/0.log"
Sep 11 03:41:10.783908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.783876    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:10.880094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:10.880011    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:11.121871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.121845    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/1.log"
Sep 11 03:41:11.122224 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.122205    2553 scope.go:117] "RemoveContainer" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"
Sep 11 03:41:11.122410 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:11.122389    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:11.457573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.457539    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:11.558617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.558580    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:11.688898 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.688870    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgkcv_7dba23ae-c972-4567-9926-d61a07546e4d/node-ca/0.log"
Sep 11 03:41:11.779191 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.779157    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:11.842153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.842121    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:11.945594 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:11.945557    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:12.329707 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.329679    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:12.539143 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.539102    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:12.539310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.539165    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:12.539310 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.539254    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:12.539379 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.539318    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:20.539302117 +0000 UTC m=+70.182092261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:12.539379 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.539254    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:12.539446 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.539390    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:20.539376664 +0000 UTC m=+70.182166805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:12.640126 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.640054    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:12.640126 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.640112    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:12.640265 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.640157    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:12.640265 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.640204    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 03:41:12.640265 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.640222    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.640208046 +0000 UTC m=+66.282998187 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:12.640265 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:12.640243    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:16.640231644 +0000 UTC m=+66.283021783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : configmap "kube-rbac-proxy" not found
Sep 11 03:41:12.690134 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:12.690105    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wjcjz_2396b115-42d4-482c-849c-cddb2ea07ba5/serve-healthcheck-canary/0.log"
Sep 11 03:41:15.002737 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.002695    2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:15.003222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.002762    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:15.003222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.003198    2553 scope.go:117] "RemoveContainer" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"
Sep 11 03:41:15.003400 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:15.003375    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:15.047100 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.047075    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:15.767747 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.767714    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:15.994061 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:15.994024    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:16.095194 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.095111    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:16.205058 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.205023    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:16.468845 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.468740    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:16.572172 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.572121    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:16.572172 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.572173    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.572227    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.572271    2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.572290    2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64748fff58-tvtgs: secret "image-registry-tls" not found
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.572315    2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.572347    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls podName:d19e9e5f-318f-439c-92f4-6adae8e9c782 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:48.57233255 +0000 UTC m=+98.215122711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls") pod "image-registry-64748fff58-tvtgs" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782") : secret "image-registry-tls" not found
Sep 11 03:41:16.572402 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.572361    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls podName:344210ec-7df8-4263-a6ca-3fda63110b03 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:48.57235543 +0000 UTC m=+98.215145570 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls") pod "dns-default-pfwn5" (UID: "344210ec-7df8-4263-a6ca-3fda63110b03") : secret "dns-default-metrics-tls" not found
Sep 11 03:41:16.575106 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.575090    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 03:41:16.583324 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.583304    2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 03:41:16.583383 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.583359    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs podName:96433f43-d834-4c3f-8d92-8cad28968754 nodeName:}" failed. No retries permitted until 2026-09-11 03:42:20.583343168 +0000 UTC m=+130.226133308 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs") pod "network-metrics-daemon-2wqbk" (UID: "96433f43-d834-4c3f-8d92-8cad28968754") : secret "metrics-daemon-secret" not found
Sep 11 03:41:16.673263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.673226    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:41:16.673437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.673289    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:16.673437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.673352    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:16.673437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.673413    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:41:16.673595 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.673434    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:16.673595 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.673485    2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 03:41:16.673595 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.673502    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:24.673483515 +0000 UTC m=+74.316273675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:16.673595 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:16.673537    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:24.673525035 +0000 UTC m=+74.316315174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : configmap "kube-rbac-proxy" not found
Sep 11 03:41:16.675775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.675756    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qffmg\" (UniqueName: \"kubernetes.io/projected/519f0793-010d-4255-89e3-c1c1c73574de-kube-api-access-qffmg\") pod \"network-check-target-pdssj\" (UID: \"519f0793-010d-4255-89e3-c1c1c73574de\") " pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:41:16.676289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.676275    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 03:41:16.686054 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.686029    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3a36fe51-02f8-4227-876f-b081ce8df153-original-pull-secret\") pod \"global-pull-secret-syncer-s5gzw\" (UID: \"3a36fe51-02f8-4227-876f-b081ce8df153\") " pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:41:16.779370 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.779345    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hqw27\""
Sep 11 03:41:16.782440 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.782425    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-s5gzw"
Sep 11 03:41:16.786849 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:16.786823    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:41:17.054755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.054676    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-s5gzw"]
Sep 11 03:41:17.057193 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.057166    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdssj"]
Sep 11 03:41:17.058067 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:17.058043    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a36fe51_02f8_4227_876f_b081ce8df153.slice/crio-c8e766842078a47698fa6124a2c38a359e6c2fea6612da7356b63622eb416c31 WatchSource:0}: Error finding container c8e766842078a47698fa6124a2c38a359e6c2fea6612da7356b63622eb416c31: Status 404 returned error can't find the container with id c8e766842078a47698fa6124a2c38a359e6c2fea6612da7356b63622eb416c31
Sep 11 03:41:17.059723 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:17.059698    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519f0793_010d_4255_89e3_c1c1c73574de.slice/crio-7bab12120b9e4e176a5b772cf8be3120610c4399806953f9541bd697f5bc8653 WatchSource:0}: Error finding container 7bab12120b9e4e176a5b772cf8be3120610c4399806953f9541bd697f5bc8653: Status 404 returned error can't find the container with id 7bab12120b9e4e176a5b772cf8be3120610c4399806953f9541bd697f5bc8653
Sep 11 03:41:17.137592 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.137551    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdssj" event={"ID":"519f0793-010d-4255-89e3-c1c1c73574de","Type":"ContainerStarted","Data":"edb6f8c30071e6f58154efec04c573e4ff3fc78ce164c541004fa45d39a5b085"}
Sep 11 03:41:17.137999 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.137659    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:41:17.137999 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.137706    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdssj" event={"ID":"519f0793-010d-4255-89e3-c1c1c73574de","Type":"ContainerStarted","Data":"7bab12120b9e4e176a5b772cf8be3120610c4399806953f9541bd697f5bc8653"}
Sep 11 03:41:17.138723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.138704    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s5gzw" event={"ID":"3a36fe51-02f8-4227-876f-b081ce8df153","Type":"ContainerStarted","Data":"c8e766842078a47698fa6124a2c38a359e6c2fea6612da7356b63622eb416c31"}
Sep 11 03:41:17.154173 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.154128    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pdssj" podStartSLOduration=66.154113794 podStartE2EDuration="1m6.154113794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:17.153644218 +0000 UTC m=+66.796434380" watchObservedRunningTime="2026-09-11 03:41:17.154113794 +0000 UTC m=+66.796903956"
Sep 11 03:41:17.466017 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.465924    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:17.870181 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:17.870151    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:18.072060 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:18.072021    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:18.269359 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:18.269316    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:18.466704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:18.466657    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:18.518210 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:18.518175    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:19.391677 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:19.391636    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:19.492756 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:19.492726    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:19.667987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:19.667908    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:19.869333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:19.869295    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:20.070859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.070806    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:20.273563 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.273508    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:20.321906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.321802    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:20.470029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.469941    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:20.606276 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.606179    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:20.606444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:20.606315    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:20.606444 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:20.606334    2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:20.606444 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:20.606405    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls podName:ddbeddec-7069-47bb-b860-7ecfade68cda nodeName:}" failed. No retries permitted until 2026-09-11 03:41:36.606388493 +0000 UTC m=+86.249178654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-bmlmg" (UID: "ddbeddec-7069-47bb-b860-7ecfade68cda") : secret "cluster-monitoring-operator-tls" not found
Sep 11 03:41:20.606444 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:20.606421    2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 03:41:20.606638 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:20.606475    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert podName:5cc12f0f-c45e-44f2-a024-97e95eebd129 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:36.606459152 +0000 UTC m=+86.249249308 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-gdd9d" (UID: "5cc12f0f-c45e-44f2-a024-97e95eebd129") : secret "networking-console-plugin-cert" not found
Sep 11 03:41:21.506740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:21.506643    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:21.866078 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:21.866035    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:22.003015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.002975    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:22.103444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.103409    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:22.158220 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.158150    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-s5gzw" event={"ID":"3a36fe51-02f8-4227-876f-b081ce8df153","Type":"ContainerStarted","Data":"28c605fe8994999c5b06d66bf5f94fc6b784bdc8a4cb019adcb5662d6bb82821"}
Sep 11 03:41:22.173711 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.173666    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-s5gzw" podStartSLOduration=67.151099225 podStartE2EDuration="1m11.173651596s" totalImagesPullingTime="4.022552371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:22.173067334 +0000 UTC m=+71.815857496" watchObservedRunningTime="2026-09-11 03:41:22.173651596 +0000 UTC m=+71.816441794"
Sep 11 03:41:22.177046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.177018    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:22.272731 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.272694    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:22.673921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:22.673889    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:23.483513 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:23.483477    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:23.675550 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:23.675520    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:24.071850 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.071797    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:24.281454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.281418    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:24.666704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.666669    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:24.735587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.735550    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:24.735998 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.735610    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:24.735998 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:24.735712    2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:24.735998 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:24.735778    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls podName:0a3fef89-bb3c-46b6-b784-6e4f91cb8e43 nodeName:}" failed. No retries permitted until 2026-09-11 03:41:40.735762926 +0000 UTC m=+90.378553066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qbbh" (UID: "0a3fef89-bb3c-46b6-b784-6e4f91cb8e43") : secret "insights-runtime-extractor-tls" not found
Sep 11 03:41:24.736119 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:24.736096    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:25.275414 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:25.275380    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:25.676831 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:25.676731    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:26.078889 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:26.078861    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:26.277405 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:26.277371    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:26.517106 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:26.517073    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:29.867037 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:29.867006    2553 scope.go:117] "RemoveContainer" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"
Sep 11 03:41:30.180139 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180064    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:41:30.180468 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180454    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/1.log"
Sep 11 03:41:30.180535 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180492    2553 generic.go:356] "Generic (PLEG): container finished" podID="44d63a06-3620-4422-aecd-45eb88efe8e9" containerID="1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8" exitCode=255
Sep 11 03:41:30.180535 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180520    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" event={"ID":"44d63a06-3620-4422-aecd-45eb88efe8e9","Type":"ContainerDied","Data":"1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8"}
Sep 11 03:41:30.180627 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180560    2553 scope.go:117] "RemoveContainer" containerID="673d57899e21a0efed3eee020b0bdc7d144c41503079a1186ce89dda1547476c"
Sep 11 03:41:30.180901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:30.180880    2553 scope.go:117] "RemoveContainer" containerID="1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8"
Sep 11 03:41:30.181064 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:30.181047    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:31.184884 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:31.184849    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:41:31.830181 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:31.830144    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:31.893647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:31.893615    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:32.117370 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.117294    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"]
Sep 11 03:41:32.120653 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.120631    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.123396 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.123378    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 03:41:32.125263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.125246    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 03:41:32.127188 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.127149    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 03:41:32.127296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.127281    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 03:41:32.128351 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.128334    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 03:41:32.128464 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.128375    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 03:41:32.128464 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.128430    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 03:41:32.138559 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.138535    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"]
Sep 11 03:41:32.192286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192258    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.192286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192288    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.192664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192323    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.192664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192431    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.192664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192500    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5zgf\" (UniqueName: \"kubernetes.io/projected/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-kube-api-access-x5zgf\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.192664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.192519    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.223758 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.223727    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:32.293001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.292971    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293160 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293030    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293160 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293103    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293275 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293173    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5zgf\" (UniqueName: \"kubernetes.io/projected/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-kube-api-access-x5zgf\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293275 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293199    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293275 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293245    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.293857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.293807    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.295424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.295396    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-ca\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.295576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.295560    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.295616 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.295580    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.296112 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.296095    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-hub\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.306403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.306378    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5zgf\" (UniqueName: \"kubernetes.io/projected/8d9b9b3a-6a42-4a08-97c4-8891e86387dd-kube-api-access-x5zgf\") pod \"cluster-proxy-proxy-agent-688fd875d8-xhjp4\" (UID: \"8d9b9b3a-6a42-4a08-97c4-8891e86387dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.307269 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.307248    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:32.405226 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.405147    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:32.436202 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.436177    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"
Sep 11 03:41:32.505468 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.505439    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:32.705846 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:32.705768    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4"]
Sep 11 03:41:32.709536 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:32.709509    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d9b9b3a_6a42_4a08_97c4_8891e86387dd.slice/crio-5223a03cd24e3d836a3ac09641b231960e7ec72e7845302f0ac911e25339ac0e WatchSource:0}: Error finding container 5223a03cd24e3d836a3ac09641b231960e7ec72e7845302f0ac911e25339ac0e: Status 404 returned error can't find the container with id 5223a03cd24e3d836a3ac09641b231960e7ec72e7845302f0ac911e25339ac0e
Sep 11 03:41:33.191234 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:33.191122    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4" event={"ID":"8d9b9b3a-6a42-4a08-97c4-8891e86387dd","Type":"ContainerStarted","Data":"5223a03cd24e3d836a3ac09641b231960e7ec72e7845302f0ac911e25339ac0e"}
Sep 11 03:41:35.002173 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:35.002138    2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:35.002657 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:35.002196    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:41:35.002657 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:35.002557    2553 scope.go:117] "RemoveContainer" containerID="1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8"
Sep 11 03:41:35.002773 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:35.002750    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:36.632655 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.632619    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:36.633346 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.632739    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:36.635580 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.635542    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ddbeddec-7069-47bb-b860-7ecfade68cda-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-bmlmg\" (UID: \"ddbeddec-7069-47bb-b860-7ecfade68cda\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:36.637060 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.637035    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5cc12f0f-c45e-44f2-a024-97e95eebd129-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-gdd9d\" (UID: \"5cc12f0f-c45e-44f2-a024-97e95eebd129\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:36.896874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.896788    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zr76h\""
Sep 11 03:41:36.902643 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.902614    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mfkns\""
Sep 11 03:41:36.904602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.904581    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"
Sep 11 03:41:36.910848 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:36.910808    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"
Sep 11 03:41:37.642230 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:37.642048    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg"]
Sep 11 03:41:37.643289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:37.643261    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d"]
Sep 11 03:41:38.722289 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:38.722253    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbeddec_7069_47bb_b860_7ecfade68cda.slice/crio-87572a64c6927fdbf7085288a5598f0585d1359f9788192db625ede5e9eb9d62 WatchSource:0}: Error finding container 87572a64c6927fdbf7085288a5598f0585d1359f9788192db625ede5e9eb9d62: Status 404 returned error can't find the container with id 87572a64c6927fdbf7085288a5598f0585d1359f9788192db625ede5e9eb9d62
Sep 11 03:41:38.723312 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:38.723284    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc12f0f_c45e_44f2_a024_97e95eebd129.slice/crio-8d1b5cc1eb4d042fbae0718f99392af25afc847d5bfb05e11560b1a55d22acfc WatchSource:0}: Error finding container 8d1b5cc1eb4d042fbae0718f99392af25afc847d5bfb05e11560b1a55d22acfc: Status 404 returned error can't find the container with id 8d1b5cc1eb4d042fbae0718f99392af25afc847d5bfb05e11560b1a55d22acfc
Sep 11 03:41:39.210290 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.210249    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4" event={"ID":"8d9b9b3a-6a42-4a08-97c4-8891e86387dd","Type":"ContainerStarted","Data":"2a17ed066acfcdda212a6278f8326b26abc5e57cc9c767b717e52d68fa8ab898"}
Sep 11 03:41:39.210481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.210300    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4" event={"ID":"8d9b9b3a-6a42-4a08-97c4-8891e86387dd","Type":"ContainerStarted","Data":"762c78b3344f0d1d764abae31c25f87b84928f0693a405173aebf84a25197e6f"}
Sep 11 03:41:39.210481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.210320    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4" event={"ID":"8d9b9b3a-6a42-4a08-97c4-8891e86387dd","Type":"ContainerStarted","Data":"954aa17de4e5fa831ab4285388821e73ded3c4b5522eceb7285c843843badcf3"}
Sep 11 03:41:39.211472 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.211449    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg" event={"ID":"ddbeddec-7069-47bb-b860-7ecfade68cda","Type":"ContainerStarted","Data":"87572a64c6927fdbf7085288a5598f0585d1359f9788192db625ede5e9eb9d62"}
Sep 11 03:41:39.212568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.212549    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d" event={"ID":"5cc12f0f-c45e-44f2-a024-97e95eebd129","Type":"ContainerStarted","Data":"8d1b5cc1eb4d042fbae0718f99392af25afc847d5bfb05e11560b1a55d22acfc"}
Sep 11 03:41:39.230201 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:39.230161    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-688fd875d8-xhjp4" podStartSLOduration=1.15413634 podStartE2EDuration="7.230146208s" totalImagesPullingTime="6.076009868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:39.229702456 +0000 UTC m=+88.872492618" watchObservedRunningTime="2026-09-11 03:41:39.230146208 +0000 UTC m=+88.872936400"
Sep 11 03:41:40.771327 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.771290    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:40.773584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.773554    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a3fef89-bb3c-46b6-b784-6e4f91cb8e43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qbbh\" (UID: \"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43\") " pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:40.792745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.792706    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:40.894391 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.894359    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:40.958006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.957980    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8h9gh\""
Sep 11 03:41:40.965870 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.965845    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9qbbh"
Sep 11 03:41:40.968310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:40.968282    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:41.222119 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.222042    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg" event={"ID":"ddbeddec-7069-47bb-b860-7ecfade68cda","Type":"ContainerStarted","Data":"a486bdeea14372184a93f62d778b2429cfce57376d42fc59cd978c28b32e3c89"}
Sep 11 03:41:41.223958 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.223923    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d" event={"ID":"5cc12f0f-c45e-44f2-a024-97e95eebd129","Type":"ContainerStarted","Data":"827ebb58944a5691c3e5a4648a78736241ca1773c4abc0b7a4d192fcbcb7f2f0"}
Sep 11 03:41:41.225975 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.225944    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:41.240182 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.240157    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9qbbh"]
Sep 11 03:41:41.242952 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.242902    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-bmlmg" podStartSLOduration=35.483890482 podStartE2EDuration="37.242883792s" totalImagesPullingTime="1.75899331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:41.242509536 +0000 UTC m=+90.885299699" watchObservedRunningTime="2026-09-11 03:41:41.242883792 +0000 UTC m=+90.885673955"
Sep 11 03:41:41.244572 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:41.244547    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3fef89_bb3c_46b6_b784_6e4f91cb8e43.slice/crio-eaccceafb086eee5683468b63d3b7b282e16f29fa2ac381ccd84e107ff0c7322 WatchSource:0}: Error finding container eaccceafb086eee5683468b63d3b7b282e16f29fa2ac381ccd84e107ff0c7322: Status 404 returned error can't find the container with id eaccceafb086eee5683468b63d3b7b282e16f29fa2ac381ccd84e107ff0c7322
Sep 11 03:41:41.258277 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.258231    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-gdd9d" podStartSLOduration=35.504337768 podStartE2EDuration="37.258214618s" totalImagesPullingTime="1.75387685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:41.257992264 +0000 UTC m=+90.900782426" watchObservedRunningTime="2026-09-11 03:41:41.258214618 +0000 UTC m=+90.901004779"
Sep 11 03:41:41.405386 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.405345    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:41.507478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:41.507403    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:42.228228 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:42.228143    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qbbh" event={"ID":"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43","Type":"ContainerStarted","Data":"2a4b102532d62eaab8b92052613811951379ff0c8a06057ec2194d393fe1951c"}
Sep 11 03:41:42.228664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:42.228246    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qbbh" event={"ID":"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43","Type":"ContainerStarted","Data":"2e0a0ab7476955558e2e8fa24f7084fbb66068c05f2f54fb76ff3d666c1d5832"}
Sep 11 03:41:42.228664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:42.228277    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qbbh" event={"ID":"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43","Type":"ContainerStarted","Data":"eaccceafb086eee5683468b63d3b7b282e16f29fa2ac381ccd84e107ff0c7322"}
Sep 11 03:41:44.236753 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:44.236717    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qbbh" event={"ID":"0a3fef89-bb3c-46b6-b784-6e4f91cb8e43","Type":"ContainerStarted","Data":"d74e76cbaee68195143db2dcd36185a12f430cf01cfb22b77736850952f48f29"}
Sep 11 03:41:44.255868 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:44.255798    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9qbbh" podStartSLOduration=34.225764397 podStartE2EDuration="36.255783482s" totalImagesPullingTime="2.030019085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 03:41:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:44.254694573 +0000 UTC m=+93.897484747" watchObservedRunningTime="2026-09-11 03:41:44.255783482 +0000 UTC m=+93.898573637"
Sep 11 03:41:45.058572 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:45.058536    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:45.119556 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:45.119522    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:45.217955 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:45.217916    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:46.866947 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:46.866918    2553 scope.go:117] "RemoveContainer" containerID="1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8"
Sep 11 03:41:46.867329 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:41:46.867069    2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-csgkk_openshift-console-operator(44d63a06-3620-4422-aecd-45eb88efe8e9)\"" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podUID="44d63a06-3620-4422-aecd-45eb88efe8e9"
Sep 11 03:41:47.390180 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:47.390139    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:48.144866 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.144838    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pdssj"
Sep 11 03:41:48.635933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.635894    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:48.636097 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.635962    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:48.638218 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.638194    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/344210ec-7df8-4263-a6ca-3fda63110b03-metrics-tls\") pod \"dns-default-pfwn5\" (UID: \"344210ec-7df8-4263-a6ca-3fda63110b03\") " pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:48.638319 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.638270    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"image-registry-64748fff58-tvtgs\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") " pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:48.884911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.884884    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2llnq\""
Sep 11 03:41:48.893059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.893012    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:48.897098 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.897084    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qlqn4\""
Sep 11 03:41:48.905267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:48.905249    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:49.168692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.168606    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:41:49.173286 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:49.173260    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19e9e5f_318f_439c_92f4_6adae8e9c782.slice/crio-6058c23324f37f3c99a344e2761bf543837cbc4bcda7ab82f0bc8191c0ab7ec9 WatchSource:0}: Error finding container 6058c23324f37f3c99a344e2761bf543837cbc4bcda7ab82f0bc8191c0ab7ec9: Status 404 returned error can't find the container with id 6058c23324f37f3c99a344e2761bf543837cbc4bcda7ab82f0bc8191c0ab7ec9
Sep 11 03:41:49.175525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.175414    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.187403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.187384    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pfwn5"]
Sep 11 03:41:49.189153 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:49.189129    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod344210ec_7df8_4263_a6ca_3fda63110b03.slice/crio-c36b898323239c83b088dfd241a693791aa97eb06f842de40b47791c8a910408 WatchSource:0}: Error finding container c36b898323239c83b088dfd241a693791aa97eb06f842de40b47791c8a910408: Status 404 returned error can't find the container with id c36b898323239c83b088dfd241a693791aa97eb06f842de40b47791c8a910408
Sep 11 03:41:49.256185 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.256153    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfwn5" event={"ID":"344210ec-7df8-4263-a6ca-3fda63110b03","Type":"ContainerStarted","Data":"c36b898323239c83b088dfd241a693791aa97eb06f842de40b47791c8a910408"}
Sep 11 03:41:49.257781 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.257755    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" event={"ID":"d19e9e5f-318f-439c-92f4-6adae8e9c782","Type":"ContainerStarted","Data":"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"}
Sep 11 03:41:49.257916 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.257801    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" event={"ID":"d19e9e5f-318f-439c-92f4-6adae8e9c782","Type":"ContainerStarted","Data":"6058c23324f37f3c99a344e2761bf543837cbc4bcda7ab82f0bc8191c0ab7ec9"}
Sep 11 03:41:49.257916 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.257883    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:41:49.278046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.277999    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" podStartSLOduration=98.27798434 podStartE2EDuration="1m38.27798434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:49.277109074 +0000 UTC m=+98.919899236" watchObservedRunningTime="2026-09-11 03:41:49.27798434 +0000 UTC m=+98.920774502"
Sep 11 03:41:49.281481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.281455    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.456910 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.456807    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.512784 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.512748    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qcr24"]
Sep 11 03:41:49.516236 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.516211    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.519251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.519230    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 03:41:49.519342 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.519266    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 03:41:49.520363 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.520348    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 03:41:49.520463 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.520346    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 03:41:49.520566 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.520353    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cv25j\""
Sep 11 03:41:49.549550 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.549524    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.644700 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644664    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.644878 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644714    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.644878 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644806    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-metrics-client-ca\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.644988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644886    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-textfile\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.644988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644951    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hwd6\" (UniqueName: \"kubernetes.io/projected/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-kube-api-access-9hwd6\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.645072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.644985    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-tls\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.645072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.645020    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-sys\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.645072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.645045    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-wtmp\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.645214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.645114    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-root\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.647479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.647452    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.745888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745775    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hwd6\" (UniqueName: \"kubernetes.io/projected/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-kube-api-access-9hwd6\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.745888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745861    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-tls\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745894    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-sys\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745925    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-wtmp\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745952    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-root\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745980    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.745983    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-sys\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746010    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746217    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-wtmp\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746311    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-metrics-client-ca\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746318    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-root\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746374    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-textfile\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746608    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746711    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-textfile\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.746906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.746771    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-metrics-client-ca\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.748212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.748179    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:49.748722 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.748702    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-tls\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.749074 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.749049    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.755763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.755743    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hwd6\" (UniqueName: \"kubernetes.io/projected/465b9ae5-e266-4485-acb7-6e0c2fc50f0a-kube-api-access-9hwd6\") pod \"node-exporter-qcr24\" (UID: \"465b9ae5-e266-4485-acb7-6e0c2fc50f0a\") " pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.825988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.825957    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcr24"
Sep 11 03:41:49.838016 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:41:49.837988    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465b9ae5_e266_4485_acb7_6e0c2fc50f0a.slice/crio-96e5c1ac50ec7b1039bd0bad5de334724699648d902d96998a23a4de80bb2273 WatchSource:0}: Error finding container 96e5c1ac50ec7b1039bd0bad5de334724699648d902d96998a23a4de80bb2273: Status 404 returned error can't find the container with id 96e5c1ac50ec7b1039bd0bad5de334724699648d902d96998a23a4de80bb2273
Sep 11 03:41:49.891779 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:49.891749    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.053468 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.053425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.155316 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.155281    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.263406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.263372    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcr24" event={"ID":"465b9ae5-e266-4485-acb7-6e0c2fc50f0a","Type":"ContainerStarted","Data":"96e5c1ac50ec7b1039bd0bad5de334724699648d902d96998a23a4de80bb2273"}
Sep 11 03:41:50.414890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.414766    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.508827 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.508745    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.618353 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.618253    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:50.952285 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:50.952248    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:52.271973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.271931    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfwn5" event={"ID":"344210ec-7df8-4263-a6ca-3fda63110b03","Type":"ContainerStarted","Data":"74c0f39106ffaf40ddd4e22c3badef1eb298869e58201b82a4cf51c2af838aa8"}
Sep 11 03:41:52.272409 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.271996    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pfwn5" event={"ID":"344210ec-7df8-4263-a6ca-3fda63110b03","Type":"ContainerStarted","Data":"f860fb5d3bb6c7e30b2c5ef2473e41502733e1a0f4e1e5616452312429123c6f"}
Sep 11 03:41:52.272409 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.272044    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:41:52.273521 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.273498    2553 generic.go:356] "Generic (PLEG): container finished" podID="465b9ae5-e266-4485-acb7-6e0c2fc50f0a" containerID="4327d6c1a22135e7b49526058063da2ae74f3321fa60f9e8e9323c692c41b4db" exitCode=0
Sep 11 03:41:52.273614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.273544    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcr24" event={"ID":"465b9ae5-e266-4485-acb7-6e0c2fc50f0a","Type":"ContainerDied","Data":"4327d6c1a22135e7b49526058063da2ae74f3321fa60f9e8e9323c692c41b4db"}
Sep 11 03:41:52.293084 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.293037    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pfwn5" podStartSLOduration=66.257512586 podStartE2EDuration="1m8.293023066s" totalImagesPullingTime="2.03551048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:52.291968351 +0000 UTC m=+101.934758523" watchObservedRunningTime="2026-09-11 03:41:52.293023066 +0000 UTC m=+101.935813227"
Sep 11 03:41:52.340429 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.340389    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:52.436680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.436636    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:52.572403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.572319    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:52.648561 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:52.648532    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:53.279356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:53.279317    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcr24" event={"ID":"465b9ae5-e266-4485-acb7-6e0c2fc50f0a","Type":"ContainerStarted","Data":"11907e8465f98b4e3db233ec2270fd41c46bd57d1ac7b42a2bb2e349d7ea89f8"}
Sep 11 03:41:53.279837 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:53.279377    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcr24" event={"ID":"465b9ae5-e266-4485-acb7-6e0c2fc50f0a","Type":"ContainerStarted","Data":"b4877dad7382de7d2c966f317e3a28338b52247b1d5c104d9cd6752f5dbfe1f0"}
Sep 11 03:41:53.299871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:53.299811    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qcr24" podStartSLOduration=2.841257571 podStartE2EDuration="4.299797387s" totalImagesPullingTime="1.381549036s" totalInitContainerRuntime="76.99078ms" isStatefulPod=true podCreationTimestamp="2026-09-11 03:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:41:53.298095528 +0000 UTC m=+102.940885689" watchObservedRunningTime="2026-09-11 03:41:53.299797387 +0000 UTC m=+102.942587548"
Sep 11 03:41:54.197118 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.197083    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:54.398770 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.398739    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:54.451675 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.451590    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:54.607633 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.607600    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:54.843211 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.843182    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:41:54.915524 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:54.915488    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:55.016438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:55.016404    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:55.592798 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:55.592768    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:55.851404 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:55.851321    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:41:55.952426 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:41:55.952394    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:01.827539 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:01.827505    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:01.867035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:01.867002    2553 scope.go:117] "RemoveContainer" containerID="1766815854b6c39e9b34e783fe12f1ec6f4fa85f0a47d0092141e12f97665cf8"
Sep 11 03:42:01.928155 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:01.928123    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:02.281627 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.281597    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pfwn5"
Sep 11 03:42:02.314130 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.314105    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:42:02.314291 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.314176    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" event={"ID":"44d63a06-3620-4422-aecd-45eb88efe8e9","Type":"ContainerStarted","Data":"bbbd0da40588be4cd27fe35d9be41462f8783c46fbf21ee2eee8b44bc44f49d9"}
Sep 11 03:42:02.314481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.314462    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:42:02.328393 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.328365    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-csgkk"
Sep 11 03:42:02.330519 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.330477    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-csgkk" podStartSLOduration=54.946393543 podStartE2EDuration="58.330462358s" totalImagesPullingTime="3.384068815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:41:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:42:02.329317698 +0000 UTC m=+111.972107861" watchObservedRunningTime="2026-09-11 03:42:02.330462358 +0000 UTC m=+111.973252520"
Sep 11 03:42:02.522541 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.522510    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:02.622704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.622611    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:02.722879 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.722846    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:02.875035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.874945    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:02.987802 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:02.987764    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:03.178776 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:03.178681    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:03.580771 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:03.580736    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:03.640417 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:03.640388    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:03.989158 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:03.989066    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:04.044357 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:04.044320    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:04.391249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:04.391210    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:04.604742 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:04.604712    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:04.727418 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:04.727319    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:04.849102 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:04.849077    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:42:05.013321 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:05.013294    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:05.067761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:05.067731    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:08.273682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:08.273647    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:08.331973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:08.331938    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:09.224031 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:09.224002    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:09.278208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:09.278169    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:10.426744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:10.426712    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:10.484519 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:10.484487    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:11.415925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:11.415888    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:12.620479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:12.620445    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:16.517810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:16.517774    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:16.619277 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:16.619244    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:16.721703 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:16.721667    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:17.345153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:17.345123    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:17.412526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:17.412495    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:19.861492 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:19.861450    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" podUID="d19e9e5f-318f-439c-92f4-6adae8e9c782" containerName="registry" containerID="cri-o://fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36" gracePeriod=30
Sep 11 03:42:20.093987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.093964    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:42:20.172291 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172211    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172291 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172259    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2wm2\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172451 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172363    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172549 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172445    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172549 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172481    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172549 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172520    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172549 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172545    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.172756 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.172568    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted\") pod \"d19e9e5f-318f-439c-92f4-6adae8e9c782\" (UID: \"d19e9e5f-318f-439c-92f4-6adae8e9c782\") "
Sep 11 03:42:20.173117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.173042    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:42:20.173117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.173062    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:42:20.175429 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.175404    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:42:20.175555 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.175429    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2" (OuterVolumeSpecName: "kube-api-access-z2wm2") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "kube-api-access-z2wm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:42:20.175555 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.175502    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:42:20.175650 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.175532    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:42:20.175650 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.175593    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:42:20.183444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.183418    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d19e9e5f-318f-439c-92f4-6adae8e9c782" (UID: "d19e9e5f-318f-439c-92f4-6adae8e9c782"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:42:20.273415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273379    2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273406    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2wm2\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-kube-api-access-z2wm2\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273415    2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d19e9e5f-318f-439c-92f4-6adae8e9c782-bound-sa-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273425    2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-image-registry-private-configuration\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273435    2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d19e9e5f-318f-439c-92f4-6adae8e9c782-installation-pull-secrets\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273444    2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-registry-certificates\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273453    2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d19e9e5f-318f-439c-92f4-6adae8e9c782-trusted-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.273662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.273463    2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d19e9e5f-318f-439c-92f4-6adae8e9c782-ca-trust-extracted\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:42:20.372878 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.372843    2553 generic.go:356] "Generic (PLEG): container finished" podID="d19e9e5f-318f-439c-92f4-6adae8e9c782" containerID="fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36" exitCode=0
Sep 11 03:42:20.372878 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.372881    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" event={"ID":"d19e9e5f-318f-439c-92f4-6adae8e9c782","Type":"ContainerDied","Data":"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"}
Sep 11 03:42:20.373067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.372921    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64748fff58-tvtgs" event={"ID":"d19e9e5f-318f-439c-92f4-6adae8e9c782","Type":"ContainerDied","Data":"6058c23324f37f3c99a344e2761bf543837cbc4bcda7ab82f0bc8191c0ab7ec9"}
Sep 11 03:42:20.373067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.372942    2553 scope.go:117] "RemoveContainer" containerID="fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"
Sep 11 03:42:20.373067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.372944    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64748fff58-tvtgs"
Sep 11 03:42:20.380918 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.380767    2553 scope.go:117] "RemoveContainer" containerID="fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"
Sep 11 03:42:20.381188 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:42:20.381161    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36\": container with ID starting with fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36 not found: ID does not exist" containerID="fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"
Sep 11 03:42:20.381262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.381193    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36"} err="failed to get container status \"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36\": rpc error: code = NotFound desc = could not find container \"fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36\": container with ID starting with fb627821106ef04beb63d6d4432d80cce5bde053414397b610dafaabbededc36 not found: ID does not exist"
Sep 11 03:42:20.393294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.393273    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:42:20.396690 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.396671    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64748fff58-tvtgs"]
Sep 11 03:42:20.410500 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.410462    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:20.604525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.604495    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:42:20.604764 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.604752    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19e9e5f-318f-439c-92f4-6adae8e9c782" containerName="registry"
Sep 11 03:42:20.604807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.604766    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19e9e5f-318f-439c-92f4-6adae8e9c782" containerName="registry"
Sep 11 03:42:20.604863 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.604826    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19e9e5f-318f-439c-92f4-6adae8e9c782" containerName="registry"
Sep 11 03:42:20.608478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.608456    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.611125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.611101    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 03:42:20.611215 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.611109    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 03:42:20.612336 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612317    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 03:42:20.612424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612365    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 03:42:20.612424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612381    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x2hwp\""
Sep 11 03:42:20.612424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612413    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 03:42:20.612577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612437    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 03:42:20.612577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.612380    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 03:42:20.621493 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.619409    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:42:20.622051 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.621639    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 03:42:20.675265 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675233    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675271    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675288    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675320    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675338    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675353    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69ldn\" (UniqueName: \"kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675376    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.675655 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.675530    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.677544 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.677526    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96433f43-d834-4c3f-8d92-8cad28968754-metrics-certs\") pod \"network-metrics-daemon-2wqbk\" (UID: \"96433f43-d834-4c3f-8d92-8cad28968754\") " pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:42:20.689518 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.689499    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sfchv\""
Sep 11 03:42:20.696849 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.696833    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wqbk"
Sep 11 03:42:20.776365 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776327    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776365 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776365    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776588 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776391    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776588 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776487    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776587    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776636    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69ldn\" (UniqueName: \"kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.776692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.776684    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.777408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.777305    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.777408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.777374    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.777662 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.777643    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.777804 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.777784    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.779256 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.779233    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.779334 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.779322    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.785301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.785283    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69ldn\" (UniqueName: \"kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn\") pod \"console-66bc9d8f44-gjdjg\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") " pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.870413 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.870327    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19e9e5f-318f-439c-92f4-6adae8e9c782" path="/var/lib/kubelet/pods/d19e9e5f-318f-439c-92f4-6adae8e9c782/volumes"
Sep 11 03:42:20.922012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.921978    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:20.962216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:20.962174    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wqbk"]
Sep 11 03:42:20.964164 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:42:20.964135    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96433f43_d834_4c3f_8d92_8cad28968754.slice/crio-1df34aff0747263f2615a93138cb03e1983533cf29a58efe663d420be0029376 WatchSource:0}: Error finding container 1df34aff0747263f2615a93138cb03e1983533cf29a58efe663d420be0029376: Status 404 returned error can't find the container with id 1df34aff0747263f2615a93138cb03e1983533cf29a58efe663d420be0029376
Sep 11 03:42:21.192110 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:21.191915    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:42:21.194396 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:42:21.194365    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8eff0a5_2189_43f8_9f86_cdede9f9f3ad.slice/crio-01eed98bbf4936f72be889ffddbaf5ae454b5920e4dc8a3805122f10c3a81181 WatchSource:0}: Error finding container 01eed98bbf4936f72be889ffddbaf5ae454b5920e4dc8a3805122f10c3a81181: Status 404 returned error can't find the container with id 01eed98bbf4936f72be889ffddbaf5ae454b5920e4dc8a3805122f10c3a81181
Sep 11 03:42:21.378734 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:21.378699    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66bc9d8f44-gjdjg" event={"ID":"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad","Type":"ContainerStarted","Data":"01eed98bbf4936f72be889ffddbaf5ae454b5920e4dc8a3805122f10c3a81181"}
Sep 11 03:42:21.379796 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:21.379771    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wqbk" event={"ID":"96433f43-d834-4c3f-8d92-8cad28968754","Type":"ContainerStarted","Data":"1df34aff0747263f2615a93138cb03e1983533cf29a58efe663d420be0029376"}
Sep 11 03:42:22.224141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.224110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:22.325194 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.325110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:22.386385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.386348    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wqbk" event={"ID":"96433f43-d834-4c3f-8d92-8cad28968754","Type":"ContainerStarted","Data":"dcf2859bde8285f07b0c9f7f3fba212fb172780823449c368359c5dc07e0baf6"}
Sep 11 03:42:22.386559 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.386410    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wqbk" event={"ID":"96433f43-d834-4c3f-8d92-8cad28968754","Type":"ContainerStarted","Data":"a829025cace5c5c291e9118f5b8a6d3ccd813a2e1bab7b28965c8edc0ac38d3c"}
Sep 11 03:42:22.403159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.403108    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2wqbk" podStartSLOduration=131.333791436 podStartE2EDuration="2m12.403090003s" totalImagesPullingTime="1.069298567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:42:22.401563798 +0000 UTC m=+132.044353959" watchObservedRunningTime="2026-09-11 03:42:22.403090003 +0000 UTC m=+132.045880173"
Sep 11 03:42:22.483061 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.483016    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:22.545935 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:22.545894    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:23.982339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:23.982296    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:24.082994 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:24.082958    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:24.138323 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:24.138235    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:24.274670 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:24.274636    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:24.400272 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:24.400181    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66bc9d8f44-gjdjg" event={"ID":"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad","Type":"ContainerStarted","Data":"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"}
Sep 11 03:42:24.417885 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:24.417808    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66bc9d8f44-gjdjg" podStartSLOduration=1.739207561 podStartE2EDuration="4.417794449s" totalImagesPullingTime="2.678586888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:42:24.41600132 +0000 UTC m=+134.058791479" watchObservedRunningTime="2026-09-11 03:42:24.417794449 +0000 UTC m=+134.060584680"
Sep 11 03:42:25.159680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:25.159638    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:25.260202 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:25.260160    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:29.666612 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:29.666578    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:29.725223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:29.725191    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:30.922568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:30.922535    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:30.922960 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:30.922588    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:30.927382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:30.927358    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:31.428160 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:31.428133    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:42:31.562048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:31.562015    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:31.612891 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:31.612855    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:31.828512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:31.828474    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:31.882512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:31.882479    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:32.111040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:32.110962    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:32.212799 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:32.212750    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:46.820028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:46.819986    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:46.887211 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:46.887169    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:42:50.547070 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:42:50.547031    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:05.480608 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:05.480510    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:05.581141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:05.581098    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:14.698682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.698641    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:43:14.701658 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.701634    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.711020 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.710995    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:43:14.754041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.754005    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:14.812769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.812732    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:14.868396 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868359    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868408    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868464    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8grh\" (UniqueName: \"kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868494    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868576    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868621    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.868724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.868646    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.961062 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.960982    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:14.970017 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.969990    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8grh\" (UniqueName: \"kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970079 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970023    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970079 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970045    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970079 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970064    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970088    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970160    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970184    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.970919 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970891    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.971048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970891    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.971048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.970934    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.971159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.971071    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.972569 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.972540    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.972663 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.972639    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:14.977742 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:14.977724    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8grh\" (UniqueName: \"kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh\") pod \"console-76fbf4865d-q7f9v\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") " pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:15.012376 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.012345    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:15.061179 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.061006    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:15.280534 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.280502    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:43:15.283434 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:43:15.283401    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e7fdfd_b431_4b76_9477_6a894fa35fe6.slice/crio-ad569e00dde7eaf76baee54d4e4cfe3cea3ba9ffa74bc09b47c1f10e2f931ed0 WatchSource:0}: Error finding container ad569e00dde7eaf76baee54d4e4cfe3cea3ba9ffa74bc09b47c1f10e2f931ed0: Status 404 returned error can't find the container with id ad569e00dde7eaf76baee54d4e4cfe3cea3ba9ffa74bc09b47c1f10e2f931ed0
Sep 11 03:43:15.504547 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.504508    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:15.559427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.559340    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:15.575226 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.575193    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76fbf4865d-q7f9v" event={"ID":"35e7fdfd-b431-4b76-9477-6a894fa35fe6","Type":"ContainerStarted","Data":"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"}
Sep 11 03:43:15.575368 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.575241    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76fbf4865d-q7f9v" event={"ID":"35e7fdfd-b431-4b76-9477-6a894fa35fe6","Type":"ContainerStarted","Data":"ad569e00dde7eaf76baee54d4e4cfe3cea3ba9ffa74bc09b47c1f10e2f931ed0"}
Sep 11 03:43:15.596500 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:15.596453    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76fbf4865d-q7f9v" podStartSLOduration=1.596438695 podStartE2EDuration="1.596438695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:43:15.595150063 +0000 UTC m=+185.237940226" watchObservedRunningTime="2026-09-11 03:43:15.596438695 +0000 UTC m=+185.239228856"
Sep 11 03:43:17.324951 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:17.324914    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:17.425876 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:17.425839    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:25.013132 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.013075    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:25.013629 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.013157    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:25.017705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.017682    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:25.613369 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.613343    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:43:25.661997 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.661966    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:43:25.761379 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:25.761351    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:31.841423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:31.841390    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:31.931419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:31.931381    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:31.981199 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:31.981165    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:43:50.685799 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:50.685740    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66bc9d8f44-gjdjg" podUID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" containerName="console" containerID="cri-o://bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817" gracePeriod=15
Sep 11 03:43:50.920273 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:50.920248    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:43:51.035454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035427    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69ldn\" (UniqueName: \"kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035628 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035470    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035628 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035507    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035628 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035534    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035747    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035843    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035874    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert\") pod \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\" (UID: \"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad\") "
Sep 11 03:43:51.035906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035873    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:43:51.035906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035891    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config" (OuterVolumeSpecName: "console-config") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:43:51.036090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.035949    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:43:51.036208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.036185    2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-config\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.036208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.036212    2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-oauth-serving-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.036392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.036224    2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-trusted-ca-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.036392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.036191    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:43:51.038019 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.037999    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:43:51.038197 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.038181    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:43:51.038254 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.038231    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn" (OuterVolumeSpecName: "kube-api-access-69ldn") pod "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" (UID: "b8eff0a5-2189-43f8-9f86-cdede9f9f3ad"). InnerVolumeSpecName "kube-api-access-69ldn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:43:51.137322 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.137286    2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.137322 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.137315    2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-serving-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.137322 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.137325    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69ldn\" (UniqueName: \"kubernetes.io/projected/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-kube-api-access-69ldn\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.137537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.137334    2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad-console-oauth-config\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:43:51.698212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.698170    2553 generic.go:356] "Generic (PLEG): container finished" podID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" containerID="bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817" exitCode=0
Sep 11 03:43:51.698581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.698219    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66bc9d8f44-gjdjg"
Sep 11 03:43:51.698581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.698237    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66bc9d8f44-gjdjg" event={"ID":"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad","Type":"ContainerDied","Data":"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"}
Sep 11 03:43:51.698581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.698286    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66bc9d8f44-gjdjg" event={"ID":"b8eff0a5-2189-43f8-9f86-cdede9f9f3ad","Type":"ContainerDied","Data":"01eed98bbf4936f72be889ffddbaf5ae454b5920e4dc8a3805122f10c3a81181"}
Sep 11 03:43:51.698581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.698314    2553 scope.go:117] "RemoveContainer" containerID="bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"
Sep 11 03:43:51.706741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.706722    2553 scope.go:117] "RemoveContainer" containerID="bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"
Sep 11 03:43:51.707009 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:43:51.706990    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817\": container with ID starting with bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817 not found: ID does not exist" containerID="bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"
Sep 11 03:43:51.707159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.707016    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817"} err="failed to get container status \"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817\": rpc error: code = NotFound desc = could not find container \"bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817\": container with ID starting with bddb4feee6359584a8ae18a54f0f35a4579222b4e2582649fa04041b98a4e817 not found: ID does not exist"
Sep 11 03:43:51.721040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.721015    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:43:51.724602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:51.724580    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66bc9d8f44-gjdjg"]
Sep 11 03:43:52.873452 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:43:52.873416    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" path="/var/lib/kubelet/pods/b8eff0a5-2189-43f8-9f86-cdede9f9f3ad/volumes"
Sep 11 03:44:05.016096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:05.016048    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:05.116648 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:05.116611    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:11.482393 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:11.482358    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:11.576809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:11.576776    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:12.136903 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.136866    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"]
Sep 11 03:44:12.137217 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.137184    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" containerName="console"
Sep 11 03:44:12.137217 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.137199    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" containerName="console"
Sep 11 03:44:12.137294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.137247    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8eff0a5-2189-43f8-9f86-cdede9f9f3ad" containerName="console"
Sep 11 03:44:12.141319 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.141300    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.144358 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.144329    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 03:44:12.144473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.144329    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 03:44:12.144473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.144380    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fs8bn\""
Sep 11 03:44:12.152043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.152020    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"]
Sep 11 03:44:12.287980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.287939    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66b7\" (UniqueName: \"kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.288163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.288045    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.288163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.288066    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.389298 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.389208    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.389298 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.389249    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.389298 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.389270    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66b7\" (UniqueName: \"kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.389645 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.389626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.389682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.389626    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.397619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.397586    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66b7\" (UniqueName: \"kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.450333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.450304    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:12.717440 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.717411    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"]
Sep 11 03:44:12.720032 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:44:12.720005    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a2c27c0_4354_4f70_a027_965e231a9509.slice/crio-c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb WatchSource:0}: Error finding container c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb: Status 404 returned error can't find the container with id c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb
Sep 11 03:44:12.773275 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:12.773240    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerStarted","Data":"c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb"}
Sep 11 03:44:19.801904 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:19.801868    2553 generic.go:356] "Generic (PLEG): container finished" podID="4a2c27c0-4354-4f70-a027-965e231a9509" containerID="eb14d691243a5ff5c947cb9f662d3fe340bd8086185ddb78859b62ac986308f2" exitCode=0
Sep 11 03:44:19.802306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:19.801921    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerDied","Data":"eb14d691243a5ff5c947cb9f662d3fe340bd8086185ddb78859b62ac986308f2"}
Sep 11 03:44:21.813216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:21.813185    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerStarted","Data":"04e1b4a99ff8885f39d56adf567a829cafeb7d4a060135f60e822dc88a7f77b1"}
Sep 11 03:44:22.818581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:22.818548    2553 generic.go:356] "Generic (PLEG): container finished" podID="4a2c27c0-4354-4f70-a027-965e231a9509" containerID="04e1b4a99ff8885f39d56adf567a829cafeb7d4a060135f60e822dc88a7f77b1" exitCode=0
Sep 11 03:44:22.818971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:22.818598    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerDied","Data":"04e1b4a99ff8885f39d56adf567a829cafeb7d4a060135f60e822dc88a7f77b1"}
Sep 11 03:44:29.845773 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:29.845736    2553 generic.go:356] "Generic (PLEG): container finished" podID="4a2c27c0-4354-4f70-a027-965e231a9509" containerID="a62a06ba1c048e21832b5b23d0687bf8381d05e41ab023c92a53e578f9dce214" exitCode=0
Sep 11 03:44:29.846156 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:29.845840    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerDied","Data":"a62a06ba1c048e21832b5b23d0687bf8381d05e41ab023c92a53e578f9dce214"}
Sep 11 03:44:30.978533 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:30.978506    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:31.142734 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.142642    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util\") pod \"4a2c27c0-4354-4f70-a027-965e231a9509\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") "
Sep 11 03:44:31.142734 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.142701    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h66b7\" (UniqueName: \"kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7\") pod \"4a2c27c0-4354-4f70-a027-965e231a9509\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") "
Sep 11 03:44:31.142984 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.142748    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle\") pod \"4a2c27c0-4354-4f70-a027-965e231a9509\" (UID: \"4a2c27c0-4354-4f70-a027-965e231a9509\") "
Sep 11 03:44:31.143412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.143386    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle" (OuterVolumeSpecName: "bundle") pod "4a2c27c0-4354-4f70-a027-965e231a9509" (UID: "4a2c27c0-4354-4f70-a027-965e231a9509"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:44:31.144958 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.144934    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7" (OuterVolumeSpecName: "kube-api-access-h66b7") pod "4a2c27c0-4354-4f70-a027-965e231a9509" (UID: "4a2c27c0-4354-4f70-a027-965e231a9509"). InnerVolumeSpecName "kube-api-access-h66b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:44:31.147397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.147374    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util" (OuterVolumeSpecName: "util") pod "4a2c27c0-4354-4f70-a027-965e231a9509" (UID: "4a2c27c0-4354-4f70-a027-965e231a9509"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:44:31.244115 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.244081    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:31.244115 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.244109    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h66b7\" (UniqueName: \"kubernetes.io/projected/4a2c27c0-4354-4f70-a027-965e231a9509-kube-api-access-h66b7\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:31.244115 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.244120    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a2c27c0-4354-4f70-a027-965e231a9509-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:31.858611 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.858579    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr" event={"ID":"4a2c27c0-4354-4f70-a027-965e231a9509","Type":"ContainerDied","Data":"c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb"}
Sep 11 03:44:31.858611 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.858618    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pd6nr"
Sep 11 03:44:31.858829 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:31.858626    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4f915f801a6603aca5290068a37a6f4266e9de64a90c03499750a8bb370cdfb"
Sep 11 03:44:33.130394 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:33.130304    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:34.003236 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:34.003207    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:35.085875 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.085839    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"]
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086167    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="util"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086184    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="util"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086198    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="pull"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086205    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="pull"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086225    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="extract"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086234    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="extract"
Sep 11 03:44:35.086416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.086295    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2c27c0-4354-4f70-a027-965e231a9509" containerName="extract"
Sep 11 03:44:35.132978 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.132951    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"]
Sep 11 03:44:35.133125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.133059    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.135739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.135713    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 11 03:44:35.135870 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.135713    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-c8b4v\""
Sep 11 03:44:35.135870 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.135860    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 11 03:44:35.273132 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.273088    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ae26a05-a04c-4942-9763-03ae65d73772-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.273306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.273152    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wm2f\" (UniqueName: \"kubernetes.io/projected/8ae26a05-a04c-4942-9763-03ae65d73772-kube-api-access-5wm2f\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.373890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.373787    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ae26a05-a04c-4942-9763-03ae65d73772-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.373890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.373869    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wm2f\" (UniqueName: \"kubernetes.io/projected/8ae26a05-a04c-4942-9763-03ae65d73772-kube-api-access-5wm2f\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.374251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.374228    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ae26a05-a04c-4942-9763-03ae65d73772-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.382487 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.382457    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wm2f\" (UniqueName: \"kubernetes.io/projected/8ae26a05-a04c-4942-9763-03ae65d73772-kube-api-access-5wm2f\") pod \"cert-manager-operator-controller-manager-55c444bf86-sf7dd\" (UID: \"8ae26a05-a04c-4942-9763-03ae65d73772\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.441971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.441940    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"
Sep 11 03:44:35.737541 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.737518    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd"]
Sep 11 03:44:35.739441 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:44:35.739406    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae26a05_a04c_4942_9763_03ae65d73772.slice/crio-da46bee5fca1b73fb705f53f9b5e902f59472c00fd8cedb4e963970528868155 WatchSource:0}: Error finding container da46bee5fca1b73fb705f53f9b5e902f59472c00fd8cedb4e963970528868155: Status 404 returned error can't find the container with id da46bee5fca1b73fb705f53f9b5e902f59472c00fd8cedb4e963970528868155
Sep 11 03:44:35.874470 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:35.874429    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd" event={"ID":"8ae26a05-a04c-4942-9763-03ae65d73772","Type":"ContainerStarted","Data":"da46bee5fca1b73fb705f53f9b5e902f59472c00fd8cedb4e963970528868155"}
Sep 11 03:44:36.744584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:36.744545    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:36.847001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:36.846963    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:38.889609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:38.889571    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd" event={"ID":"8ae26a05-a04c-4942-9763-03ae65d73772","Type":"ContainerStarted","Data":"f693f7288f0a2d9bdabfaf4c488d829f2ff322ca45c7bef639ad679a5638f0dc"}
Sep 11 03:44:38.913308 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:38.913254    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-sf7dd" podStartSLOduration=1.178815295 podStartE2EDuration="3.913239552s" totalImagesPullingTime="2.734424257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:44:38.911186093 +0000 UTC m=+268.553976379" watchObservedRunningTime="2026-09-11 03:44:38.913239552 +0000 UTC m=+268.556029714"
Sep 11 03:44:39.205087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.205003    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:39.310381 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.310346    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:39.361425 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.361387    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"]
Sep 11 03:44:39.363658 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.363643    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.366416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.366393    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 03:44:39.366573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.366554    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 03:44:39.367769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.367755    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fs8bn\""
Sep 11 03:44:39.374876 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.374853    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"]
Sep 11 03:44:39.508775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.508688    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.508775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.508727    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6dzv\" (UniqueName: \"kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.508775 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.508754    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.609690 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.609655    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.609898 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.609745    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.609898 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.609780    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6dzv\" (UniqueName: \"kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.610060 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.610041    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.610125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.610084    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.625402 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.625374    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6dzv\" (UniqueName: \"kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.673219 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.673187    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:39.944462 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:39.944438    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"]
Sep 11 03:44:39.946787 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:44:39.946761    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d31b2a_4e5b_4ad4_8345_45053344684b.slice/crio-a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66 WatchSource:0}: Error finding container a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66: Status 404 returned error can't find the container with id a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66
Sep 11 03:44:40.898688 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:40.898607    2553 generic.go:356] "Generic (PLEG): container finished" podID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerID="e0335c8f942976543e64cf5e5996af97681d23b1ae51f63e0173149c477ee2ac" exitCode=0
Sep 11 03:44:40.898688 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:40.898652    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552" event={"ID":"d2d31b2a-4e5b-4ad4-8345-45053344684b","Type":"ContainerDied","Data":"e0335c8f942976543e64cf5e5996af97681d23b1ae51f63e0173149c477ee2ac"}
Sep 11 03:44:40.898909 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:40.898689    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552" event={"ID":"d2d31b2a-4e5b-4ad4-8345-45053344684b","Type":"ContainerStarted","Data":"a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66"}
Sep 11 03:44:41.585806 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:41.585771    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:41.788198 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:41.788148    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:43.534356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:43.534322    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:43.635267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:43.635230    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:43.912882 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:43.912780    2553 generic.go:356] "Generic (PLEG): container finished" podID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerID="b269249519e76a1f5e554f92dcdd361721bbbb4bcdadb42898fcbc9f78a38149" exitCode=0
Sep 11 03:44:43.912882 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:43.912853    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552" event={"ID":"d2d31b2a-4e5b-4ad4-8345-45053344684b","Type":"ContainerDied","Data":"b269249519e76a1f5e554f92dcdd361721bbbb4bcdadb42898fcbc9f78a38149"}
Sep 11 03:44:44.918593 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:44.918559    2553 generic.go:356] "Generic (PLEG): container finished" podID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerID="b9149bcb04c9ea0406e78149c992c9b47b82dc2f674568fb1a2f409769219627" exitCode=0
Sep 11 03:44:44.918990 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:44.918626    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552" event={"ID":"d2d31b2a-4e5b-4ad4-8345-45053344684b","Type":"ContainerDied","Data":"b9149bcb04c9ea0406e78149c992c9b47b82dc2f674568fb1a2f409769219627"}
Sep 11 03:44:46.040243 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.040219    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:46.058377 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.058351    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util\") pod \"d2d31b2a-4e5b-4ad4-8345-45053344684b\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") "
Sep 11 03:44:46.058515 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.058399    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle\") pod \"d2d31b2a-4e5b-4ad4-8345-45053344684b\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") "
Sep 11 03:44:46.058515 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.058439    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6dzv\" (UniqueName: \"kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv\") pod \"d2d31b2a-4e5b-4ad4-8345-45053344684b\" (UID: \"d2d31b2a-4e5b-4ad4-8345-45053344684b\") "
Sep 11 03:44:46.058852 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.058799    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle" (OuterVolumeSpecName: "bundle") pod "d2d31b2a-4e5b-4ad4-8345-45053344684b" (UID: "d2d31b2a-4e5b-4ad4-8345-45053344684b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:44:46.060640 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.060618    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv" (OuterVolumeSpecName: "kube-api-access-d6dzv") pod "d2d31b2a-4e5b-4ad4-8345-45053344684b" (UID: "d2d31b2a-4e5b-4ad4-8345-45053344684b"). InnerVolumeSpecName "kube-api-access-d6dzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:44:46.063769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.063744    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util" (OuterVolumeSpecName: "util") pod "d2d31b2a-4e5b-4ad4-8345-45053344684b" (UID: "d2d31b2a-4e5b-4ad4-8345-45053344684b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:44:46.159647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.159611    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:46.159647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.159639    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2d31b2a-4e5b-4ad4-8345-45053344684b-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:46.159647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.159648    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6dzv\" (UniqueName: \"kubernetes.io/projected/d2d31b2a-4e5b-4ad4-8345-45053344684b-kube-api-access-d6dzv\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:44:46.927709 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.927671    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552" event={"ID":"d2d31b2a-4e5b-4ad4-8345-45053344684b","Type":"ContainerDied","Data":"a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66"}
Sep 11 03:44:46.927880 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.927713    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbt552"
Sep 11 03:44:46.927880 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:46.927743    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a36deb21205e7e1a1e6601ba05092c51bae23e5ac0715e2dc4a933f9243d9d66"
Sep 11 03:44:49.180248 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.180216    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:49.518953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.518915    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xc7p9"]
Sep 11 03:44:49.519263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519222    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="extract"
Sep 11 03:44:49.519263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519245    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="extract"
Sep 11 03:44:49.519403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519265    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="pull"
Sep 11 03:44:49.519403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519274    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="pull"
Sep 11 03:44:49.519403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519299    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="util"
Sep 11 03:44:49.519403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519308    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="util"
Sep 11 03:44:49.519403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.519379    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d31b2a-4e5b-4ad4-8345-45053344684b" containerName="extract"
Sep 11 03:44:49.523268 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.523248    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.525745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.525716    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 03:44:49.527124 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.527103    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-scjq6\""
Sep 11 03:44:49.527239 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.527109    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 03:44:49.531532 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.531513    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xc7p9"]
Sep 11 03:44:49.581779 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.581745    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dsng\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-kube-api-access-6dsng\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.581947 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.581786    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-bound-sa-token\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.682641 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.682598    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dsng\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-kube-api-access-6dsng\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.682789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.682647    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-bound-sa-token\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.691535 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.691509    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-bound-sa-token\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.691595 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.691553    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dsng\" (UniqueName: \"kubernetes.io/projected/dfd93d44-fed0-4823-9534-1526e9597c53-kube-api-access-6dsng\") pod \"cert-manager-55b4d954d5-xc7p9\" (UID: \"dfd93d44-fed0-4823-9534-1526e9597c53\") " pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:49.832969 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:49.832886    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xc7p9"
Sep 11 03:44:50.099109 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:50.099029    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xc7p9"]
Sep 11 03:44:50.102064 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:44:50.102028    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfd93d44_fed0_4823_9534_1526e9597c53.slice/crio-b5717cd99e70758530b36bed401060b1850582935016a6326ccf2ec20836e9d7 WatchSource:0}: Error finding container b5717cd99e70758530b36bed401060b1850582935016a6326ccf2ec20836e9d7: Status 404 returned error can't find the container with id b5717cd99e70758530b36bed401060b1850582935016a6326ccf2ec20836e9d7
Sep 11 03:44:50.944012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:50.943961    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xc7p9" event={"ID":"dfd93d44-fed0-4823-9534-1526e9597c53","Type":"ContainerStarted","Data":"b5717cd99e70758530b36bed401060b1850582935016a6326ccf2ec20836e9d7"}
Sep 11 03:44:52.953875 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:52.953840    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xc7p9" event={"ID":"dfd93d44-fed0-4823-9534-1526e9597c53","Type":"ContainerStarted","Data":"99f8c0673d1cec96708d0994528fa8c678f0348755658cfbdc526a59b5c4a6d2"}
Sep 11 03:44:52.971797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:52.971733    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xc7p9" podStartSLOduration=1.6282071949999999 podStartE2EDuration="3.971718112s" totalImagesPullingTime="2.343510917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:44:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:44:52.97025342 +0000 UTC m=+282.613043582" watchObservedRunningTime="2026-09-11 03:44:52.971718112 +0000 UTC m=+282.614508273"
Sep 11 03:44:53.352275 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:53.352235    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:53.445184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:53.445144    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:55.050577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.050542    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"]
Sep 11 03:44:55.053742 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.053710    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.056253 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.056232    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 03:44:55.056344 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.056298    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-fn2wj\""
Sep 11 03:44:55.057292 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.057274    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 03:44:55.061404 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.061384    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"]
Sep 11 03:44:55.127552 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.127520    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064de959-1713-4003-98c8-aa2cf405ec34-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.127727 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.127579    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnsrr\" (UniqueName: \"kubernetes.io/projected/064de959-1713-4003-98c8-aa2cf405ec34-kube-api-access-bnsrr\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.227925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.227893    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnsrr\" (UniqueName: \"kubernetes.io/projected/064de959-1713-4003-98c8-aa2cf405ec34-kube-api-access-bnsrr\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.228080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.228052    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064de959-1713-4003-98c8-aa2cf405ec34-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.228420 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.228391    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/064de959-1713-4003-98c8-aa2cf405ec34-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.235925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.235904    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnsrr\" (UniqueName: \"kubernetes.io/projected/064de959-1713-4003-98c8-aa2cf405ec34-kube-api-access-bnsrr\") pod \"openshift-lws-operator-fd8ccff4c-rs2jk\" (UID: \"064de959-1713-4003-98c8-aa2cf405ec34\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.363252 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.363169    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"
Sep 11 03:44:55.631647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.631624    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk"]
Sep 11 03:44:55.633991 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:44:55.633961    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod064de959_1713_4003_98c8_aa2cf405ec34.slice/crio-903fdc3f00b2858fd5702699cb2593fa2efa50600db3ca16e0133c0a42c01baa WatchSource:0}: Error finding container 903fdc3f00b2858fd5702699cb2593fa2efa50600db3ca16e0133c0a42c01baa: Status 404 returned error can't find the container with id 903fdc3f00b2858fd5702699cb2593fa2efa50600db3ca16e0133c0a42c01baa
Sep 11 03:44:55.664619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.664588    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:55.966781 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:55.966699    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk" event={"ID":"064de959-1713-4003-98c8-aa2cf405ec34","Type":"ContainerStarted","Data":"903fdc3f00b2858fd5702699cb2593fa2efa50600db3ca16e0133c0a42c01baa"}
Sep 11 03:44:57.976619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:57.976580    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk" event={"ID":"064de959-1713-4003-98c8-aa2cf405ec34","Type":"ContainerStarted","Data":"859260c920ebe34185dfcc5160ef8c8133d0c774b3dea5645993628eff669b9d"}
Sep 11 03:44:57.992541 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:57.992457    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rs2jk" podStartSLOduration=1.35526661 podStartE2EDuration="2.992438949s" totalImagesPullingTime="1.637172339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:44:57.991804667 +0000 UTC m=+287.634594844" watchObservedRunningTime="2026-09-11 03:44:57.992438949 +0000 UTC m=+287.635229110"
Sep 11 03:44:58.773420 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:58.773385    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:44:58.879600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:44:58.879565    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:00.432627 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.432592    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"]
Sep 11 03:45:00.435776 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.435760    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.438647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.438617    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 03:45:00.438647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.438623    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 03:45:00.439716 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.439700    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fs8bn\""
Sep 11 03:45:00.444262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.444238    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"]
Sep 11 03:45:00.463591 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.463561    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.463696 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.463598    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.463736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.463685    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl86h\" (UniqueName: \"kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.564515 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.564474    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.564680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.564523    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.564680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.564577    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl86h\" (UniqueName: \"kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.564955 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.564934    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.564955 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.564947    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.574117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.574091    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl86h\" (UniqueName: \"kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.745247 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.745171    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:00.774563 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:00.774532    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:01.022375 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:01.022353    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"]
Sep 11 03:45:01.024290 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:01.024265    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208c4a47_fb60_495d_bd3a_9872cbd6a0fb.slice/crio-fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620 WatchSource:0}: Error finding container fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620: Status 404 returned error can't find the container with id fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620
Sep 11 03:45:01.993568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:01.993485    2553 generic.go:356] "Generic (PLEG): container finished" podID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerID="1fa64d2db7ccae0f594457a31c9a7f5b32464f1ea77c11941731a612bafbbe7a" exitCode=0
Sep 11 03:45:01.993933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:01.993580    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c" event={"ID":"208c4a47-fb60-495d-bd3a-9872cbd6a0fb","Type":"ContainerDied","Data":"1fa64d2db7ccae0f594457a31c9a7f5b32464f1ea77c11941731a612bafbbe7a"}
Sep 11 03:45:01.993933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:01.993637    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c" event={"ID":"208c4a47-fb60-495d-bd3a-9872cbd6a0fb","Type":"ContainerStarted","Data":"fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620"}
Sep 11 03:45:04.002758 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:04.002723    2553 generic.go:356] "Generic (PLEG): container finished" podID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerID="c3318154f7f5c7eee99dc85d593d85bdc251ebc27e968c719bd80562936fa880" exitCode=0
Sep 11 03:45:04.003199 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:04.002798    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c" event={"ID":"208c4a47-fb60-495d-bd3a-9872cbd6a0fb","Type":"ContainerDied","Data":"c3318154f7f5c7eee99dc85d593d85bdc251ebc27e968c719bd80562936fa880"}
Sep 11 03:45:05.009303 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:05.009268    2553 generic.go:356] "Generic (PLEG): container finished" podID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerID="b0e73ffcb955356743605587d1d9810004ad53fd26ff75e2f70dc8ea325fcc2e" exitCode=0
Sep 11 03:45:05.009707 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:05.009349    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c" event={"ID":"208c4a47-fb60-495d-bd3a-9872cbd6a0fb","Type":"ContainerDied","Data":"b0e73ffcb955356743605587d1d9810004ad53fd26ff75e2f70dc8ea325fcc2e"}
Sep 11 03:45:06.133136 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.133103    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:06.209087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.209050    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle\") pod \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") "
Sep 11 03:45:06.209282 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.209109    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util\") pod \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") "
Sep 11 03:45:06.209282 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.209155    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl86h\" (UniqueName: \"kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h\") pod \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\" (UID: \"208c4a47-fb60-495d-bd3a-9872cbd6a0fb\") "
Sep 11 03:45:06.209763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.209738    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle" (OuterVolumeSpecName: "bundle") pod "208c4a47-fb60-495d-bd3a-9872cbd6a0fb" (UID: "208c4a47-fb60-495d-bd3a-9872cbd6a0fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:45:06.211207 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.211181    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h" (OuterVolumeSpecName: "kube-api-access-nl86h") pod "208c4a47-fb60-495d-bd3a-9872cbd6a0fb" (UID: "208c4a47-fb60-495d-bd3a-9872cbd6a0fb"). InnerVolumeSpecName "kube-api-access-nl86h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:45:06.214572 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.214536    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util" (OuterVolumeSpecName: "util") pod "208c4a47-fb60-495d-bd3a-9872cbd6a0fb" (UID: "208c4a47-fb60-495d-bd3a-9872cbd6a0fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:45:06.310415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.310323    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:06.310415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.310355    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:06.310415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:06.310369    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl86h\" (UniqueName: \"kubernetes.io/projected/208c4a47-fb60-495d-bd3a-9872cbd6a0fb-kube-api-access-nl86h\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:07.019663 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:07.019631    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c" event={"ID":"208c4a47-fb60-495d-bd3a-9872cbd6a0fb","Type":"ContainerDied","Data":"fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620"}
Sep 11 03:45:07.019846 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:07.019681    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe929b24de7bd28e3c3d953912ed010322bcc86e8dda124e4be6787c17f56620"
Sep 11 03:45:07.019846 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:07.019638    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7mp9c"
Sep 11 03:45:07.257378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:07.257287    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:07.307636 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:07.307604    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:09.274043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:09.274005    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:09.332102 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:09.332067    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:10.781611 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:10.781585    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:45:10.782422 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:10.782399    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:45:10.792634 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:10.792610    2553 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 03:45:11.949730 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.949696    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"]
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.949997    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="pull"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950012    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="pull"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950026    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="extract"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950031    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="extract"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950043    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="util"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950049    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="util"
Sep 11 03:45:11.950940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.950099    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="208c4a47-fb60-495d-bd3a-9872cbd6a0fb" containerName="extract"
Sep 11 03:45:11.951719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.951704    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:11.954269 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.954243    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fs8bn\""
Sep 11 03:45:11.954407 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.954278    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 03:45:11.954407 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.954243    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 03:45:11.963688 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:11.963665    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"]
Sep 11 03:45:12.051867 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.051834    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.051867 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.051867    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.052076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.051897    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mspq\" (UniqueName: \"kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.152512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.152467    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mspq\" (UniqueName: \"kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.152685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.152560    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.152685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.152584    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.153028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.153005    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.153028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.153020    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.160478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.160457    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mspq\" (UniqueName: \"kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.260427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.260342    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:12.532969 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:12.532907    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"]
Sep 11 03:45:12.534960 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:12.534928    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c38ee0_cde9_4356_a551_f5bb3ac58f84.slice/crio-e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28 WatchSource:0}: Error finding container e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28: Status 404 returned error can't find the container with id e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28
Sep 11 03:45:13.046630 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.046595    2553 generic.go:356] "Generic (PLEG): container finished" podID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerID="9c15e614d1228d76a4175555037cd3c1bc17946456a324d5302e641914b610f7" exitCode=0
Sep 11 03:45:13.047030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.046676    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v" event={"ID":"f8c38ee0-cde9-4356-a551-f5bb3ac58f84","Type":"ContainerDied","Data":"9c15e614d1228d76a4175555037cd3c1bc17946456a324d5302e641914b610f7"}
Sep 11 03:45:13.047030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.046725    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v" event={"ID":"f8c38ee0-cde9-4356-a551-f5bb3ac58f84","Type":"ContainerStarted","Data":"e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28"}
Sep 11 03:45:13.047136 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.047122    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 03:45:13.059126 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.059103    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:13.154870 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.154831    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:13.670139 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.670102    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:13.902195 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:13.902160    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.004013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.003923    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.042978 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.042952    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"]
Sep 11 03:45:14.044761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.044744    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.049438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.049416    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 03:45:14.049779 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.049597    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 03:45:14.049779 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.049674    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 03:45:14.049901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.049789    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8t266\""
Sep 11 03:45:14.049901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.049877    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 03:45:14.053272 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.053248    2553 generic.go:356] "Generic (PLEG): container finished" podID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerID="71fcc021d7cbac92c063637defa3f06f730aadca3ec2c0e40bf76c6790ed840c" exitCode=0
Sep 11 03:45:14.053356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.053326    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v" event={"ID":"f8c38ee0-cde9-4356-a551-f5bb3ac58f84","Type":"ContainerDied","Data":"71fcc021d7cbac92c063637defa3f06f730aadca3ec2c0e40bf76c6790ed840c"}
Sep 11 03:45:14.061624 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.061602    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"]
Sep 11 03:45:14.167579 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.167535    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.169589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.169550    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-manifests\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.169689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.169614    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.169689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.169656    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-tmp\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.169689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.169678    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.169869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.169743    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvvc\" (UniqueName: \"kubernetes.io/projected/afd69650-85d4-492c-a0a9-4e72c32d2634-kube-api-access-wjvvc\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.270540 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.270509    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.270702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.270557    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-tmp\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.270702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.270582    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.270702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.270617    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvvc\" (UniqueName: \"kubernetes.io/projected/afd69650-85d4-492c-a0a9-4e72c32d2634-kube-api-access-wjvvc\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.270702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.270648    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-manifests\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.271113 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.271086    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-manifests\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.272973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.272944    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/afd69650-85d4-492c-a0a9-4e72c32d2634-tmp\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.273142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.273121    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.273292 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.273268    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/afd69650-85d4-492c-a0a9-4e72c32d2634-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.278726 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.278703    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvvc\" (UniqueName: \"kubernetes.io/projected/afd69650-85d4-492c-a0a9-4e72c32d2634-kube-api-access-wjvvc\") pod \"opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q\" (UID: \"afd69650-85d4-492c-a0a9-4e72c32d2634\") " pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.287047 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.287022    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.355263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.355236    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:14.387076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.387032    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.490676 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.490642    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.587945 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.587859    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:14.634050 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.634021    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"]
Sep 11 03:45:14.637737 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:14.637606    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd69650_85d4_492c_a0a9_4e72c32d2634.slice/crio-fe2ccfdb36ee702cb3a668591bd17b99471e43a53e8f3adb0a1a0a2f15a42945 WatchSource:0}: Error finding container fe2ccfdb36ee702cb3a668591bd17b99471e43a53e8f3adb0a1a0a2f15a42945: Status 404 returned error can't find the container with id fe2ccfdb36ee702cb3a668591bd17b99471e43a53e8f3adb0a1a0a2f15a42945
Sep 11 03:45:14.686857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:14.686803    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:15.059410 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:15.059375    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q" event={"ID":"afd69650-85d4-492c-a0a9-4e72c32d2634","Type":"ContainerStarted","Data":"fe2ccfdb36ee702cb3a668591bd17b99471e43a53e8f3adb0a1a0a2f15a42945"}
Sep 11 03:45:15.061419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:15.061391    2553 generic.go:356] "Generic (PLEG): container finished" podID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerID="c8cf4c07993c5c8e0576970bacf682d5e8c13324089e6f6ab8ed36eb629e98c1" exitCode=0
Sep 11 03:45:15.061545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:15.061481    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v" event={"ID":"f8c38ee0-cde9-4356-a551-f5bb3ac58f84","Type":"ContainerDied","Data":"c8cf4c07993c5c8e0576970bacf682d5e8c13324089e6f6ab8ed36eb629e98c1"}
Sep 11 03:45:17.622739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.622714    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:17.702976 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.702955    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle\") pod \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") "
Sep 11 03:45:17.703069 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.703004    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util\") pod \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") "
Sep 11 03:45:17.703069 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.703035    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mspq\" (UniqueName: \"kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq\") pod \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\" (UID: \"f8c38ee0-cde9-4356-a551-f5bb3ac58f84\") "
Sep 11 03:45:17.703605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.703583    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle" (OuterVolumeSpecName: "bundle") pod "f8c38ee0-cde9-4356-a551-f5bb3ac58f84" (UID: "f8c38ee0-cde9-4356-a551-f5bb3ac58f84"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:45:17.704865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.704845    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq" (OuterVolumeSpecName: "kube-api-access-4mspq") pod "f8c38ee0-cde9-4356-a551-f5bb3ac58f84" (UID: "f8c38ee0-cde9-4356-a551-f5bb3ac58f84"). InnerVolumeSpecName "kube-api-access-4mspq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:45:17.707795 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.707772    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util" (OuterVolumeSpecName: "util") pod "f8c38ee0-cde9-4356-a551-f5bb3ac58f84" (UID: "f8c38ee0-cde9-4356-a551-f5bb3ac58f84"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:45:17.804092 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.804059    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mspq\" (UniqueName: \"kubernetes.io/projected/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-kube-api-access-4mspq\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:17.804092 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.804085    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:17.804092 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:17.804095    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8c38ee0-cde9-4356-a551-f5bb3ac58f84-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:18.078026 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:18.077999    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v"
Sep 11 03:45:18.078206 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:18.078038    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b05gx6v" event={"ID":"f8c38ee0-cde9-4356-a551-f5bb3ac58f84","Type":"ContainerDied","Data":"e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28"}
Sep 11 03:45:18.078206 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:18.078085    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0494f7b9cf962ee1e051991ebd7279c8d480b77b37c04d55ebc1eef512ece28"
Sep 11 03:45:18.079693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:18.079668    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q" event={"ID":"afd69650-85d4-492c-a0a9-4e72c32d2634","Type":"ContainerStarted","Data":"96c29f38da945e64fcf9e6076b59f11ec7299824deab93229e39a7b4e30a05f3"}
Sep 11 03:45:19.085395 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:19.085365    2553 generic.go:356] "Generic (PLEG): container finished" podID="afd69650-85d4-492c-a0a9-4e72c32d2634" containerID="96c29f38da945e64fcf9e6076b59f11ec7299824deab93229e39a7b4e30a05f3" exitCode=0
Sep 11 03:45:19.085789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:19.085443    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q" event={"ID":"afd69650-85d4-492c-a0a9-4e72c32d2634","Type":"ContainerDied","Data":"96c29f38da945e64fcf9e6076b59f11ec7299824deab93229e39a7b4e30a05f3"}
Sep 11 03:45:20.091362 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:20.091321    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q" event={"ID":"afd69650-85d4-492c-a0a9-4e72c32d2634","Type":"ContainerStarted","Data":"20c113f22ad458c5ec1d946d406c7ffd76eaaa8c6fe9b18c957f0f25bf7c99fb"}
Sep 11 03:45:20.091780 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:20.091421    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:20.112810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:20.112768    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q" podStartSLOduration=1.661384899 podStartE2EDuration="6.112754756s" totalImagesPullingTime="3.405970651s" totalInitContainerRuntime="1.045399206s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:20.111581367 +0000 UTC m=+309.754371528" watchObservedRunningTime="2026-09-11 03:45:20.112754756 +0000 UTC m=+309.755544901"
Sep 11 03:45:20.969435 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:20.969401    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:21.070614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:21.070573    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:24.716830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:24.716775    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:24.819548 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:24.819504    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:31.097942 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:31.097911    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q"
Sep 11 03:45:33.363552 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:33.363515    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:34.937373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:34.937339    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.241143 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.241106    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.264618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264586    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"]
Sep 11 03:45:44.264888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264873    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="extract"
Sep 11 03:45:44.264934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264889    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="extract"
Sep 11 03:45:44.264934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264903    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="util"
Sep 11 03:45:44.264934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264908    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="util"
Sep 11 03:45:44.264934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264919    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="pull"
Sep 11 03:45:44.264934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264924    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="pull"
Sep 11 03:45:44.265086 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.264972    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c38ee0-cde9-4356-a551-f5bb3ac58f84" containerName="extract"
Sep 11 03:45:44.269152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.269134    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.273985 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.273964    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 03:45:44.274084 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.274029    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-c2754\""
Sep 11 03:45:44.274697 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.274671    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 03:45:44.274697 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.274693    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 03:45:44.274848 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.274678    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 03:45:44.281571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.281547    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"]
Sep 11 03:45:44.310158 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.310130    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthfn\" (UniqueName: \"kubernetes.io/projected/ae641e21-ecf2-43d8-a172-e08d8da54e74-kube-api-access-gthfn\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.310296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.310225    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae641e21-ecf2-43d8-a172-e08d8da54e74-tls-certs\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.310296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.310252    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae641e21-ecf2-43d8-a172-e08d8da54e74-tmp\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.346435 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.346401    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9fscw"]
Sep 11 03:45:44.349985 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.349960    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.352805 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.352785    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 03:45:44.352989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.352785    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-m7qvz\""
Sep 11 03:45:44.354388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.354365    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9fscw"]
Sep 11 03:45:44.372316 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.372290    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.411146 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.411117    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1290a39-9e24-4ebf-8d61-70e9bbad7590-cert\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.411294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.411151    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92xq\" (UniqueName: \"kubernetes.io/projected/e1290a39-9e24-4ebf-8d61-70e9bbad7590-kube-api-access-q92xq\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.411294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.411182    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae641e21-ecf2-43d8-a172-e08d8da54e74-tls-certs\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.411294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.411243    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae641e21-ecf2-43d8-a172-e08d8da54e74-tmp\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.411465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.411300    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthfn\" (UniqueName: \"kubernetes.io/projected/ae641e21-ecf2-43d8-a172-e08d8da54e74-kube-api-access-gthfn\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.413411 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.413380    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae641e21-ecf2-43d8-a172-e08d8da54e74-tmp\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.413582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.413562    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae641e21-ecf2-43d8-a172-e08d8da54e74-tls-certs\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.421753 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.421734    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthfn\" (UniqueName: \"kubernetes.io/projected/ae641e21-ecf2-43d8-a172-e08d8da54e74-kube-api-access-gthfn\") pod \"kube-auth-proxy-86748cc5bf-frg7j\" (UID: \"ae641e21-ecf2-43d8-a172-e08d8da54e74\") " pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.468947 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.468910    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.512155 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.512129    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1290a39-9e24-4ebf-8d61-70e9bbad7590-cert\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.512328 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.512173    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92xq\" (UniqueName: \"kubernetes.io/projected/e1290a39-9e24-4ebf-8d61-70e9bbad7590-kube-api-access-q92xq\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.514632 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.514606    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1290a39-9e24-4ebf-8d61-70e9bbad7590-cert\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.525423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.525396    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92xq\" (UniqueName: \"kubernetes.io/projected/e1290a39-9e24-4ebf-8d61-70e9bbad7590-kube-api-access-q92xq\") pod \"odh-model-controller-7998c67dc4-9fscw\" (UID: \"e1290a39-9e24-4ebf-8d61-70e9bbad7590\") " pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.579289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.579247    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"
Sep 11 03:45:44.581776 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.581742    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.663359 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.663307    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:44.801901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.801797    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6jngc"]
Sep 11 03:45:44.805177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.805155    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.807665 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.807639    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 03:45:44.807923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.807902    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-9l4rs\""
Sep 11 03:45:44.809881 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.809844    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.814384 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.814168    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzh4b\" (UniqueName: \"kubernetes.io/projected/01c0b92a-72df-401d-be77-d5656b6accaa-kube-api-access-wzh4b\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.814384 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.814267    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b92a-72df-401d-be77-d5656b6accaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.815545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.815523    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6jngc"]
Sep 11 03:45:44.849839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.849803    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j"]
Sep 11 03:45:44.853022 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:44.852986    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae641e21_ecf2_43d8_a172_e08d8da54e74.slice/crio-be553234375a60524c54f61fefda9fcecc5d4a06e1476ce467287bf9bbe3c49e WatchSource:0}: Error finding container be553234375a60524c54f61fefda9fcecc5d4a06e1476ce467287bf9bbe3c49e: Status 404 returned error can't find the container with id be553234375a60524c54f61fefda9fcecc5d4a06e1476ce467287bf9bbe3c49e
Sep 11 03:45:44.887643 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.887603    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:44.915389 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.915359    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b92a-72df-401d-be77-d5656b6accaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.915570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.915416    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzh4b\" (UniqueName: \"kubernetes.io/projected/01c0b92a-72df-401d-be77-d5656b6accaa-kube-api-access-wzh4b\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.918101 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.918079    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01c0b92a-72df-401d-be77-d5656b6accaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.926944 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.926921    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzh4b\" (UniqueName: \"kubernetes.io/projected/01c0b92a-72df-401d-be77-d5656b6accaa-kube-api-access-wzh4b\") pod \"model-serving-api-54d89bd5fc-6jngc\" (UID: \"01c0b92a-72df-401d-be77-d5656b6accaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:44.940208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:44.940178    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9fscw"]
Sep 11 03:45:44.942287 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:44.942251    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1290a39_9e24_4ebf_8d61_70e9bbad7590.slice/crio-9c351e275fd92401582598384b57859f911f8e7be7f51dc4a1b15feca55ebd2b WatchSource:0}: Error finding container 9c351e275fd92401582598384b57859f911f8e7be7f51dc4a1b15feca55ebd2b: Status 404 returned error can't find the container with id 9c351e275fd92401582598384b57859f911f8e7be7f51dc4a1b15feca55ebd2b
Sep 11 03:45:45.116936 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.116858    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:45.196035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.196000    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw" event={"ID":"e1290a39-9e24-4ebf-8d61-70e9bbad7590","Type":"ContainerStarted","Data":"9c351e275fd92401582598384b57859f911f8e7be7f51dc4a1b15feca55ebd2b"}
Sep 11 03:45:45.197168 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.197146    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j" event={"ID":"ae641e21-ecf2-43d8-a172-e08d8da54e74","Type":"ContainerStarted","Data":"be553234375a60524c54f61fefda9fcecc5d4a06e1476ce467287bf9bbe3c49e"}
Sep 11 03:45:45.391865 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.391840    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6jngc"]
Sep 11 03:45:45.393663 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:45.393635    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c0b92a_72df_401d_be77_d5656b6accaa.slice/crio-e20662846246b3549ac77243989f6044352f48107fa92540dc1d5b5680febf9c WatchSource:0}: Error finding container e20662846246b3549ac77243989f6044352f48107fa92540dc1d5b5680febf9c: Status 404 returned error can't find the container with id e20662846246b3549ac77243989f6044352f48107fa92540dc1d5b5680febf9c
Sep 11 03:45:45.413066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.413042    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:45.415857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.415830    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:45.425488 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.425455    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.428448 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.428419    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 03:45:45.428558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.428452    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 03:45:45.428716 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.428683    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 03:45:45.428829 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.428782    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 03:45:45.429359 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.429339    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-l9kcw\""
Sep 11 03:45:45.429858 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.429838    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:45.478933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.478905    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:45.522016 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.521981    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522016 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522028    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwgcp\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522061    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522100    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522149    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522181    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.522262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.522252    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623383    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623450    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623479    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623542    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623574    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623601    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwgcp\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.623952 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.623624    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.625002 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.624303    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.626367 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.626319    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.626747 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.626725    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.627462 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.627417    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.627661 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.627634    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.633322 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.633250    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.633322 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.633305    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwgcp\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp\") pod \"istiod-openshift-gateway-7b7995d5cd-j6bll\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:45.735716 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:45.735628    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:46.042495 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:46.042459    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:46.053454 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:46.052852    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6113027f_750f_4f97_a0f9_5e91a50e3c33.slice/crio-31f356041cdb89fa08ce2e0cc97ca2c10c670c144bc0673b14041bca57d05119 WatchSource:0}: Error finding container 31f356041cdb89fa08ce2e0cc97ca2c10c670c144bc0673b14041bca57d05119: Status 404 returned error can't find the container with id 31f356041cdb89fa08ce2e0cc97ca2c10c670c144bc0673b14041bca57d05119
Sep 11 03:45:46.206988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:46.206952    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" event={"ID":"6113027f-750f-4f97-a0f9-5e91a50e3c33","Type":"ContainerStarted","Data":"31f356041cdb89fa08ce2e0cc97ca2c10c670c144bc0673b14041bca57d05119"}
Sep 11 03:45:46.209498 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:46.209468    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc" event={"ID":"01c0b92a-72df-401d-be77-d5656b6accaa","Type":"ContainerStarted","Data":"e20662846246b3549ac77243989f6044352f48107fa92540dc1d5b5680febf9c"}
Sep 11 03:45:46.300736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:46.300620    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:51.613575 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:51.613534    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:45:51.613853 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:51.613618    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:45:52.254663 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.254619    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc" event={"ID":"01c0b92a-72df-401d-be77-d5656b6accaa","Type":"ContainerStarted","Data":"d0c0ad42c5dfece88f6b96a0718c923413c59b3c8dd99fa1d78e43bf7cafc889"}
Sep 11 03:45:52.255458 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.255433    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:45:52.257613 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.257579    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw" event={"ID":"e1290a39-9e24-4ebf-8d61-70e9bbad7590","Type":"ContainerStarted","Data":"539cc6c1547d989bc5f5440ebb824699d0c18d15a4b353efbbf891fbbe5aeb38"}
Sep 11 03:45:52.257719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.257645    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:45:52.259852 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.259791    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" event={"ID":"6113027f-750f-4f97-a0f9-5e91a50e3c33","Type":"ContainerStarted","Data":"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"}
Sep 11 03:45:52.260002 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.259946    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:52.262373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.262294    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j" event={"ID":"ae641e21-ecf2-43d8-a172-e08d8da54e74","Type":"ContainerStarted","Data":"09a22e2e9917517c1b30046dc9158d64bfd7b13d4401b43c3ac7f900ffd2380a"}
Sep 11 03:45:52.262373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.262345    2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-j6bll container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 03:45:52.262545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.262394    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 03:45:52.275526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.275466    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc" podStartSLOduration=2.126082667 podStartE2EDuration="8.275446344s" totalImagesPullingTime="6.149363677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:52.272905531 +0000 UTC m=+341.915695694" watchObservedRunningTime="2026-09-11 03:45:52.275446344 +0000 UTC m=+341.918236507"
Sep 11 03:45:52.301057 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.301000    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86748cc5bf-frg7j" podStartSLOduration=1.6120117139999999 podStartE2EDuration="8.300966143s" totalImagesPullingTime="6.688954429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:52.293307351 +0000 UTC m=+341.936097506" watchObservedRunningTime="2026-09-11 03:45:52.300966143 +0000 UTC m=+341.943756305"
Sep 11 03:45:52.317600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.317554    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" podStartSLOduration=1.7610345760000001 podStartE2EDuration="7.317540714s" totalImagesPullingTime="5.556506138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:52.316182503 +0000 UTC m=+341.958972666" watchObservedRunningTime="2026-09-11 03:45:52.317540714 +0000 UTC m=+341.960330876"
Sep 11 03:45:52.331402 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.331352    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw" podStartSLOduration=1.7299653259999999 podStartE2EDuration="8.331334928s" totalImagesPullingTime="6.601369602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:52.329907636 +0000 UTC m=+341.972697811" watchObservedRunningTime="2026-09-11 03:45:52.331334928 +0000 UTC m=+341.974125092"
Sep 11 03:45:52.704186 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.704147    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:52.803998 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:52.803962    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:53.102051 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.102015    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"]
Sep 11 03:45:53.105456 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.105436    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.108961 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.108931    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7dl6s\""
Sep 11 03:45:53.120582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.120559    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"]
Sep 11 03:45:53.154443 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.154414    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:53.192647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192619    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.192797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192656    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.192797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192683    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.192797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192712    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.192797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192751    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhpwj\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-kube-api-access-lhpwj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.192797 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192788    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.193008 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192806    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.193008 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192859    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.193008 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.192893    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.268525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.268499    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:53.293569 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293532    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.293710 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293639    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.293710 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293696    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.293833 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293725    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.293833 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293755    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhpwj\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-kube-api-access-lhpwj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.293940 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.293858    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294198    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294205    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294235    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294289    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294309    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294372    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294684 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294584    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.294786 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.294765    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.296928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.296907    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.297213 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.297194    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.312089 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.312062    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.312682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.312657    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhpwj\" (UniqueName: \"kubernetes.io/projected/bb4fa34b-5792-44f6-836d-9c1ae137df9e-kube-api-access-lhpwj\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cmhct8\" (UID: \"bb4fa34b-5792-44f6-836d-9c1ae137df9e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.342283 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.342255    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:53.415698 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.415625    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:53.443432 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.443401    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:53.695708 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:53.695675    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"]
Sep 11 03:45:53.696089 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:53.696066    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4fa34b_5792_44f6_836d_9c1ae137df9e.slice/crio-52c498145afd19e47535ca95b4565d5983b9ba6b18a5f97aa0eadf0b955b0a20 WatchSource:0}: Error finding container 52c498145afd19e47535ca95b4565d5983b9ba6b18a5f97aa0eadf0b955b0a20: Status 404 returned error can't find the container with id 52c498145afd19e47535ca95b4565d5983b9ba6b18a5f97aa0eadf0b955b0a20
Sep 11 03:45:54.273835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:54.273787    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8" event={"ID":"bb4fa34b-5792-44f6-836d-9c1ae137df9e","Type":"ContainerStarted","Data":"52c498145afd19e47535ca95b4565d5983b9ba6b18a5f97aa0eadf0b955b0a20"}
Sep 11 03:45:55.872740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:55.872701    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:55.989295 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:55.989254    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"]
Sep 11 03:45:55.996476 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:55.996450    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.001690 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.001579    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 03:45:56.002335 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.002311    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4btjp\""
Sep 11 03:45:56.009447 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.009422    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"]
Sep 11 03:45:56.012741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.012719    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.016335 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.016313    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d7qbd\""
Sep 11 03:45:56.016437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.016339    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 11 03:45:56.016931 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.016858    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:56.017763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.017740    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"]
Sep 11 03:45:56.045897 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.045867    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"]
Sep 11 03:45:56.122971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.122888    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a41e9d5b-4228-4516-9117-a9282bd84e5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.122971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.122942    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg6s2\" (UniqueName: \"kubernetes.io/projected/e0aacd83-19e1-4965-a283-c82b75a7f1c3-kube-api-access-pg6s2\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.123186 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.123050    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m66h\" (UniqueName: \"kubernetes.io/projected/a41e9d5b-4228-4516-9117-a9282bd84e5c-kube-api-access-6m66h\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.123186 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.123095    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0aacd83-19e1-4965-a283-c82b75a7f1c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.216996 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.216959    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:56.224062 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.224031    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m66h\" (UniqueName: \"kubernetes.io/projected/a41e9d5b-4228-4516-9117-a9282bd84e5c-kube-api-access-6m66h\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.224208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.224090    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0aacd83-19e1-4965-a283-c82b75a7f1c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.224284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.224252    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a41e9d5b-4228-4516-9117-a9282bd84e5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.224337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.224308    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg6s2\" (UniqueName: \"kubernetes.io/projected/e0aacd83-19e1-4965-a283-c82b75a7f1c3-kube-api-access-pg6s2\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.227196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.227172    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0aacd83-19e1-4965-a283-c82b75a7f1c3-cert\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.227387 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.227363    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a41e9d5b-4228-4516-9117-a9282bd84e5c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.237163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.237130    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m66h\" (UniqueName: \"kubernetes.io/projected/a41e9d5b-4228-4516-9117-a9282bd84e5c-kube-api-access-6m66h\") pod \"llmisvc-controller-manager-777c6bd7d5-b9pcn\" (UID: \"a41e9d5b-4228-4516-9117-a9282bd84e5c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.237475 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.237452    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg6s2\" (UniqueName: \"kubernetes.io/projected/e0aacd83-19e1-4965-a283-c82b75a7f1c3-kube-api-access-pg6s2\") pod \"kserve-controller-manager-d6f4b65b8-hhtzd\" (UID: \"e0aacd83-19e1-4965-a283-c82b75a7f1c3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.310238 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.310201    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:45:56.316075 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.315980    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:56.324439 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.324402    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:45:56.437624 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.437550    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:45:56.519427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.519185    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:45:56.519427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.519274    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:45:56.519427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.519321    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:45:56.716467 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.716436    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"]
Sep 11 03:45:56.719114 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:56.719072    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0aacd83_19e1_4965_a283_c82b75a7f1c3.slice/crio-3cc094439e462d2e07d1d24358eefd65b8d51ac82319fef20823e9b8b6318ef5 WatchSource:0}: Error finding container 3cc094439e462d2e07d1d24358eefd65b8d51ac82319fef20823e9b8b6318ef5: Status 404 returned error can't find the container with id 3cc094439e462d2e07d1d24358eefd65b8d51ac82319fef20823e9b8b6318ef5
Sep 11 03:45:56.720036 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:56.720014    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"]
Sep 11 03:45:56.723203 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:45:56.723178    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda41e9d5b_4228_4516_9117_a9282bd84e5c.slice/crio-e392b3eaf58953900b75fbcfa0d4a71a21faacd292cdf6b2c897ffa6ae6e6859 WatchSource:0}: Error finding container e392b3eaf58953900b75fbcfa0d4a71a21faacd292cdf6b2c897ffa6ae6e6859: Status 404 returned error can't find the container with id e392b3eaf58953900b75fbcfa0d4a71a21faacd292cdf6b2c897ffa6ae6e6859
Sep 11 03:45:57.029975 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.029940    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:57.030420 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.030154    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerName="discovery" containerID="cri-o://f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d" gracePeriod=30
Sep 11 03:45:57.279349 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.279324    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:57.292089 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.292030    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd" event={"ID":"e0aacd83-19e1-4965-a283-c82b75a7f1c3","Type":"ContainerStarted","Data":"3cc094439e462d2e07d1d24358eefd65b8d51ac82319fef20823e9b8b6318ef5"}
Sep 11 03:45:57.293699 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.293670    2553 generic.go:356] "Generic (PLEG): container finished" podID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerID="f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d" exitCode=0
Sep 11 03:45:57.293798 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.293718    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"
Sep 11 03:45:57.293798 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.293745    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" event={"ID":"6113027f-750f-4f97-a0f9-5e91a50e3c33","Type":"ContainerDied","Data":"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"}
Sep 11 03:45:57.293798 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.293788    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll" event={"ID":"6113027f-750f-4f97-a0f9-5e91a50e3c33","Type":"ContainerDied","Data":"31f356041cdb89fa08ce2e0cc97ca2c10c670c144bc0673b14041bca57d05119"}
Sep 11 03:45:57.293986 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.293834    2553 scope.go:117] "RemoveContainer" containerID="f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"
Sep 11 03:45:57.295328 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.295245    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn" event={"ID":"a41e9d5b-4228-4516-9117-a9282bd84e5c","Type":"ContainerStarted","Data":"e392b3eaf58953900b75fbcfa0d4a71a21faacd292cdf6b2c897ffa6ae6e6859"}
Sep 11 03:45:57.297378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.297353    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8" event={"ID":"bb4fa34b-5792-44f6-836d-9c1ae137df9e","Type":"ContainerStarted","Data":"4883cca794c1afcfb253b2e7bf543d1faa8ba5223f02808598610c28be179642"}
Sep 11 03:45:57.303926 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.303906    2553 scope.go:117] "RemoveContainer" containerID="f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"
Sep 11 03:45:57.304235 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:45:57.304205    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d\": container with ID starting with f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d not found: ID does not exist" containerID="f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"
Sep 11 03:45:57.304331 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.304252    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d"} err="failed to get container status \"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d\": rpc error: code = NotFound desc = could not find container \"f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d\": container with ID starting with f8cf5f514f5b9fd6ee5e4ea6746bd6abd8723d828ae8858eda8bbcd5c201ea0d not found: ID does not exist"
Sep 11 03:45:57.324420 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.324362    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8" podStartSLOduration=1.503971135 podStartE2EDuration="4.324340729s" totalImagesPullingTime="2.820369594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:45:57.323153015 +0000 UTC m=+346.965943176" watchObservedRunningTime="2026-09-11 03:45:57.324340729 +0000 UTC m=+346.967130892"
Sep 11 03:45:57.333552 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333523    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333568    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333599    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333657    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwgcp\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333673    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333899 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333691    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.333899 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.333747    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert\") pod \"6113027f-750f-4f97-a0f9-5e91a50e3c33\" (UID: \"6113027f-750f-4f97-a0f9-5e91a50e3c33\") "
Sep 11 03:45:57.334609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.334330    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:45:57.334609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.334471    2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-ca-configmap\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.336632 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.336561    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp" (OuterVolumeSpecName: "kube-api-access-fwgcp") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "kube-api-access-fwgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:45:57.337290 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.336990    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts" (OuterVolumeSpecName: "cacerts") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:45:57.337290 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.336991    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs" (OuterVolumeSpecName: "local-certs") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:45:57.337290 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.337164    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:45:57.337500 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.337398    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token" (OuterVolumeSpecName: "istio-token") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:45:57.337897 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.337878    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6113027f-750f-4f97-a0f9-5e91a50e3c33" (UID: "6113027f-750f-4f97-a0f9-5e91a50e3c33"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:45:57.416281 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.416244    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:57.418183 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.418140    2553 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 03:45:57.418329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.418200    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8" podUID="bb4fa34b-5792-44f6-836d-9c1ae137df9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 03:45:57.435334 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435300    2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-csr-dns-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.435334 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435337    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.435528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435350    2553 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-istio-kubeconfig\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.435528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435363    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwgcp\" (UniqueName: \"kubernetes.io/projected/6113027f-750f-4f97-a0f9-5e91a50e3c33-kube-api-access-fwgcp\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.435528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435376    2553 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6113027f-750f-4f97-a0f9-5e91a50e3c33-cacerts\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.435528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.435389    2553 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6113027f-750f-4f97-a0f9-5e91a50e3c33-local-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:45:57.621246 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.621216    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:57.626528 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:57.626504    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j6bll"]
Sep 11 03:45:58.420261 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:58.420225    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:58.874144 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:58.874098    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" path="/var/lib/kubelet/pods/6113027f-750f-4f97-a0f9-5e91a50e3c33/volumes"
Sep 11 03:45:59.310045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:59.309995    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:45:59.311312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:45:59.311285    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cmhct8"
Sep 11 03:46:00.315766 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.315726    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd" event={"ID":"e0aacd83-19e1-4965-a283-c82b75a7f1c3","Type":"ContainerStarted","Data":"8ffdc8067b351c3d241fadc6438eb29387652bda4f3bb0b8e80fbaf00be1d0bb"}
Sep 11 03:46:00.316201 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.315836    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:46:00.317219 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.317193    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn" event={"ID":"a41e9d5b-4228-4516-9117-a9282bd84e5c","Type":"ContainerStarted","Data":"643dee359fa4f49d21c5a4ed0c7ccb6c2cb7c86f1569a12951e4d470122c61c3"}
Sep 11 03:46:00.317477 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.317460    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:46:00.332576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.332538    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd" podStartSLOduration=2.4742719429999998 podStartE2EDuration="5.332525485s" totalImagesPullingTime="2.858253542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:46:00.331257888 +0000 UTC m=+349.974048051" watchObservedRunningTime="2026-09-11 03:46:00.332525485 +0000 UTC m=+349.975315645"
Sep 11 03:46:00.348691 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:00.348647    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn" podStartSLOduration=2.45146044 podStartE2EDuration="5.348635675s" totalImagesPullingTime="2.897175235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:45:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:46:00.347097231 +0000 UTC m=+349.989887394" watchObservedRunningTime="2026-09-11 03:46:00.348635675 +0000 UTC m=+349.991425837"
Sep 11 03:46:03.270724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:03.270638    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-9fscw"
Sep 11 03:46:04.280306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:04.280274    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6jngc"
Sep 11 03:46:31.324148 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:31.324112    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-b9pcn"
Sep 11 03:46:31.327174 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:31.327156    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hhtzd"
Sep 11 03:46:42.624175 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:42.624137    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:46:45.518921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:45.518886    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:46:45.617256 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:45.617226    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:46:49.325656 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.325620    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"]
Sep 11 03:46:49.326042 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.325952    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerName="discovery"
Sep 11 03:46:49.326042 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.325964    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerName="discovery"
Sep 11 03:46:49.326042 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.326019    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6113027f-750f-4f97-a0f9-5e91a50e3c33" containerName="discovery"
Sep 11 03:46:49.327973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.327956    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.330625 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.330603    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 03:46:49.330625 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.330615    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fs8bn\""
Sep 11 03:46:49.331725 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.331705    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 03:46:49.336237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.336210    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"]
Sep 11 03:46:49.364908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.364882    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.365009 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.364939    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ng8q\" (UniqueName: \"kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.365049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.365024    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.426868 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.426833    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"]
Sep 11 03:46:49.428974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.428959    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.438408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.438384    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"]
Sep 11 03:46:49.465964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.465935    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.466099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.465979    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.466099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466002    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.466099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466037    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.466099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466066    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ng8q\" (UniqueName: \"kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.466099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466090    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg67x\" (UniqueName: \"kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.466337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466318    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.466376 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.466355    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.474318 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.474291    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ng8q\" (UniqueName: \"kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.526433 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.526398    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"]
Sep 11 03:46:49.528710 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.528688    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.538339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.538308    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"]
Sep 11 03:46:49.567180 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567136    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.567354 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567200    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.567354 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567234    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.567354 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567275    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg67x\" (UniqueName: \"kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.567469 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567344    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qth4x\" (UniqueName: \"kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.567571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567552    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.567614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567576    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.567614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.567605    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.575989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.575919    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg67x\" (UniqueName: \"kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.630187 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.630155    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"]
Sep 11 03:46:49.632290 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.632275    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.637564 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.637527    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:49.643921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.643895    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"]
Sep 11 03:46:49.668697 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668656    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zvgs\" (UniqueName: \"kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.668921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668712    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.668921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668781    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.668921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668838    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.668921 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668895    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.669150 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.668933    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qth4x\" (UniqueName: \"kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.669198 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.669167    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.669249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.669226    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.677406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.677375    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qth4x\" (UniqueName: \"kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.737721 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.737693    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:49.770029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.769993    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zvgs\" (UniqueName: \"kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.770029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.770042    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.770230 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.770076    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.770465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.770445    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.770502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.770485    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.779378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.779344    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zvgs\" (UniqueName: \"kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:49.839052 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.838968    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:49.908837 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.908778    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"]
Sep 11 03:46:49.911266 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:46:49.911237    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010fdffc_044c_45f1_955a_d8bc022d0075.slice/crio-094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d WatchSource:0}: Error finding container 094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d: Status 404 returned error can't find the container with id 094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d
Sep 11 03:46:49.941197 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:49.941173    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:50.019249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.019217    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"]
Sep 11 03:46:50.020758 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:46:50.020727    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde078a10_9474_40a7_9ab7_cf72ba8a7993.slice/crio-7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d WatchSource:0}: Error finding container 7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d: Status 404 returned error can't find the container with id 7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d
Sep 11 03:46:50.112629 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.112605    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"]
Sep 11 03:46:50.114836 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:46:50.114788    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b26284_2596_4690_964e_b577ced1d0bd.slice/crio-5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe WatchSource:0}: Error finding container 5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe: Status 404 returned error can't find the container with id 5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe
Sep 11 03:46:50.214974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.214950    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"]
Sep 11 03:46:50.258603 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:46:50.258568    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf516d4c_ba08_4408_aec6_14b0ef49dfd8.slice/crio-2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c WatchSource:0}: Error finding container 2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c: Status 404 returned error can't find the container with id 2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c
Sep 11 03:46:50.547673 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.547638    2553 generic.go:356] "Generic (PLEG): container finished" podID="e2b26284-2596-4690-964e-b577ced1d0bd" containerID="05eaefee83bfce837d32ce0d91b441af18ed67aa49700054654345475c1d3d3d" exitCode=0
Sep 11 03:46:50.548116 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.547725    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6" event={"ID":"e2b26284-2596-4690-964e-b577ced1d0bd","Type":"ContainerDied","Data":"05eaefee83bfce837d32ce0d91b441af18ed67aa49700054654345475c1d3d3d"}
Sep 11 03:46:50.548116 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.547776    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6" event={"ID":"e2b26284-2596-4690-964e-b577ced1d0bd","Type":"ContainerStarted","Data":"5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe"}
Sep 11 03:46:50.549561 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.549478    2553 generic.go:356] "Generic (PLEG): container finished" podID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerID="5092de6fa6b16a6f561a4759bc44dd18ead10896237edb42b4dac3e6c07c0652" exitCode=0
Sep 11 03:46:50.549561 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.549513    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9" event={"ID":"de078a10-9474-40a7-9ab7-cf72ba8a7993","Type":"ContainerDied","Data":"5092de6fa6b16a6f561a4759bc44dd18ead10896237edb42b4dac3e6c07c0652"}
Sep 11 03:46:50.549561 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.549558    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9" event={"ID":"de078a10-9474-40a7-9ab7-cf72ba8a7993","Type":"ContainerStarted","Data":"7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d"}
Sep 11 03:46:50.551119 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.551094    2553 generic.go:356] "Generic (PLEG): container finished" podID="010fdffc-044c-45f1-955a-d8bc022d0075" containerID="c7d46769201512763b9b873aa5341f804b987be0dedf611c5d7eb69515b5da24" exitCode=0
Sep 11 03:46:50.551239 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.551173    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerDied","Data":"c7d46769201512763b9b873aa5341f804b987be0dedf611c5d7eb69515b5da24"}
Sep 11 03:46:50.551301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.551223    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerStarted","Data":"094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d"}
Sep 11 03:46:50.553081 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.553061    2553 generic.go:356] "Generic (PLEG): container finished" podID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerID="539d17d4d9068df641766956a4a0d9764d24487459f7d8bf6cc7b01e418f0d10" exitCode=0
Sep 11 03:46:50.553229 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.553086    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275" event={"ID":"bf516d4c-ba08-4408-aec6-14b0ef49dfd8","Type":"ContainerDied","Data":"539d17d4d9068df641766956a4a0d9764d24487459f7d8bf6cc7b01e418f0d10"}
Sep 11 03:46:50.553229 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:50.553123    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275" event={"ID":"bf516d4c-ba08-4408-aec6-14b0ef49dfd8","Type":"ContainerStarted","Data":"2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c"}
Sep 11 03:46:51.560209 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:51.560175    2553 generic.go:356] "Generic (PLEG): container finished" podID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerID="6325fbdce17c5467cf88437ca1bb1c85bde6cee0f2beb8dfb89f61db8af78a93" exitCode=0
Sep 11 03:46:51.560591 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:51.560212    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9" event={"ID":"de078a10-9474-40a7-9ab7-cf72ba8a7993","Type":"ContainerDied","Data":"6325fbdce17c5467cf88437ca1bb1c85bde6cee0f2beb8dfb89f61db8af78a93"}
Sep 11 03:46:51.562365 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:51.562342    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerStarted","Data":"da21da92f35123141c33748edf11e66b910b0ae0c3ecea6042713af02006ef48"}
Sep 11 03:46:52.569012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.568973    2553 generic.go:356] "Generic (PLEG): container finished" podID="e2b26284-2596-4690-964e-b577ced1d0bd" containerID="e6e11efd573ccf98d5cbd3f1b6aa593e4f9813591b41709736592c54fbbd106d" exitCode=0
Sep 11 03:46:52.569456 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.569062    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6" event={"ID":"e2b26284-2596-4690-964e-b577ced1d0bd","Type":"ContainerDied","Data":"e6e11efd573ccf98d5cbd3f1b6aa593e4f9813591b41709736592c54fbbd106d"}
Sep 11 03:46:52.571202 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.571174    2553 generic.go:356] "Generic (PLEG): container finished" podID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerID="e9ea25dbd977af2357a6438bb709f9fbadb9d936938902372c9657d4fd31e153" exitCode=0
Sep 11 03:46:52.571308 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.571248    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9" event={"ID":"de078a10-9474-40a7-9ab7-cf72ba8a7993","Type":"ContainerDied","Data":"e9ea25dbd977af2357a6438bb709f9fbadb9d936938902372c9657d4fd31e153"}
Sep 11 03:46:52.573028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.573005    2553 generic.go:356] "Generic (PLEG): container finished" podID="010fdffc-044c-45f1-955a-d8bc022d0075" containerID="da21da92f35123141c33748edf11e66b910b0ae0c3ecea6042713af02006ef48" exitCode=0
Sep 11 03:46:52.573117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.573085    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerDied","Data":"da21da92f35123141c33748edf11e66b910b0ae0c3ecea6042713af02006ef48"}
Sep 11 03:46:52.575312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.575290    2553 generic.go:356] "Generic (PLEG): container finished" podID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerID="91a78c347f7230389e7b79fea07b78fddcd04009730ac64c404ee9b53b9f9ca1" exitCode=0
Sep 11 03:46:52.575412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:52.575362    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275" event={"ID":"bf516d4c-ba08-4408-aec6-14b0ef49dfd8","Type":"ContainerDied","Data":"91a78c347f7230389e7b79fea07b78fddcd04009730ac64c404ee9b53b9f9ca1"}
Sep 11 03:46:53.586831 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.586787    2553 generic.go:356] "Generic (PLEG): container finished" podID="e2b26284-2596-4690-964e-b577ced1d0bd" containerID="d570dd7e03bff6e483fbf1f60c8d2c589665bfd45aa49468a95625b0db737d3d" exitCode=0
Sep 11 03:46:53.587194 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.586870    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6" event={"ID":"e2b26284-2596-4690-964e-b577ced1d0bd","Type":"ContainerDied","Data":"d570dd7e03bff6e483fbf1f60c8d2c589665bfd45aa49468a95625b0db737d3d"}
Sep 11 03:46:53.588982 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.588962    2553 generic.go:356] "Generic (PLEG): container finished" podID="010fdffc-044c-45f1-955a-d8bc022d0075" containerID="2ff54c483064ee2c67d7aa2d4c8ed70701c59313b5d15dd28d96e7d53cfb90be" exitCode=0
Sep 11 03:46:53.589072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.589031    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerDied","Data":"2ff54c483064ee2c67d7aa2d4c8ed70701c59313b5d15dd28d96e7d53cfb90be"}
Sep 11 03:46:53.590988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.590969    2553 generic.go:356] "Generic (PLEG): container finished" podID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerID="e43b57bb3e1f8e0a34052be1b9dbdbde5f166727e6f19a28c4b4fc077ee89201" exitCode=0
Sep 11 03:46:53.591077 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.591055    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275" event={"ID":"bf516d4c-ba08-4408-aec6-14b0ef49dfd8","Type":"ContainerDied","Data":"e43b57bb3e1f8e0a34052be1b9dbdbde5f166727e6f19a28c4b4fc077ee89201"}
Sep 11 03:46:53.716681 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.716658    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:53.807677 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.807639    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util\") pod \"de078a10-9474-40a7-9ab7-cf72ba8a7993\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") "
Sep 11 03:46:53.807864 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.807687    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle\") pod \"de078a10-9474-40a7-9ab7-cf72ba8a7993\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") "
Sep 11 03:46:53.807864 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.807800    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg67x\" (UniqueName: \"kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x\") pod \"de078a10-9474-40a7-9ab7-cf72ba8a7993\" (UID: \"de078a10-9474-40a7-9ab7-cf72ba8a7993\") "
Sep 11 03:46:53.808173 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.808148    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle" (OuterVolumeSpecName: "bundle") pod "de078a10-9474-40a7-9ab7-cf72ba8a7993" (UID: "de078a10-9474-40a7-9ab7-cf72ba8a7993"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:53.809836 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.809796    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x" (OuterVolumeSpecName: "kube-api-access-jg67x") pod "de078a10-9474-40a7-9ab7-cf72ba8a7993" (UID: "de078a10-9474-40a7-9ab7-cf72ba8a7993"). InnerVolumeSpecName "kube-api-access-jg67x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:46:53.813070 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.813034    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util" (OuterVolumeSpecName: "util") pod "de078a10-9474-40a7-9ab7-cf72ba8a7993" (UID: "de078a10-9474-40a7-9ab7-cf72ba8a7993"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:53.908724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.908654    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg67x\" (UniqueName: \"kubernetes.io/projected/de078a10-9474-40a7-9ab7-cf72ba8a7993-kube-api-access-jg67x\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:53.908724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.908679    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:53.908724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:53.908689    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de078a10-9474-40a7-9ab7-cf72ba8a7993-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.598284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.598243    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9" event={"ID":"de078a10-9474-40a7-9ab7-cf72ba8a7993","Type":"ContainerDied","Data":"7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d"}
Sep 11 03:46:54.598284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.598282    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7vq7m9"
Sep 11 03:46:54.598686 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.598298    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e2f2e50fc7cd12f23a1bc05e387344925fdab10e4c22962fedba8288159979d"
Sep 11 03:46:54.744304 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.744282    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:54.787859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.787835    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:54.790616 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.790595    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:54.816147 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816125    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qth4x\" (UniqueName: \"kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x\") pod \"e2b26284-2596-4690-964e-b577ced1d0bd\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") "
Sep 11 03:46:54.816251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816155    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util\") pod \"e2b26284-2596-4690-964e-b577ced1d0bd\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") "
Sep 11 03:46:54.816251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816211    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util\") pod \"010fdffc-044c-45f1-955a-d8bc022d0075\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") "
Sep 11 03:46:54.816251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816228    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle\") pod \"e2b26284-2596-4690-964e-b577ced1d0bd\" (UID: \"e2b26284-2596-4690-964e-b577ced1d0bd\") "
Sep 11 03:46:54.816427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816270    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ng8q\" (UniqueName: \"kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q\") pod \"010fdffc-044c-45f1-955a-d8bc022d0075\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") "
Sep 11 03:46:54.816427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.816307    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle\") pod \"010fdffc-044c-45f1-955a-d8bc022d0075\" (UID: \"010fdffc-044c-45f1-955a-d8bc022d0075\") "
Sep 11 03:46:54.817043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.817007    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle" (OuterVolumeSpecName: "bundle") pod "010fdffc-044c-45f1-955a-d8bc022d0075" (UID: "010fdffc-044c-45f1-955a-d8bc022d0075"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:54.817147 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.817086    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle" (OuterVolumeSpecName: "bundle") pod "e2b26284-2596-4690-964e-b577ced1d0bd" (UID: "e2b26284-2596-4690-964e-b577ced1d0bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:54.818320 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.818295    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x" (OuterVolumeSpecName: "kube-api-access-qth4x") pod "e2b26284-2596-4690-964e-b577ced1d0bd" (UID: "e2b26284-2596-4690-964e-b577ced1d0bd"). InnerVolumeSpecName "kube-api-access-qth4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:46:54.818750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.818726    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q" (OuterVolumeSpecName: "kube-api-access-4ng8q") pod "010fdffc-044c-45f1-955a-d8bc022d0075" (UID: "010fdffc-044c-45f1-955a-d8bc022d0075"). InnerVolumeSpecName "kube-api-access-4ng8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:46:54.822132 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.822107    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util" (OuterVolumeSpecName: "util") pod "010fdffc-044c-45f1-955a-d8bc022d0075" (UID: "010fdffc-044c-45f1-955a-d8bc022d0075"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:54.822246 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.822228    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util" (OuterVolumeSpecName: "util") pod "e2b26284-2596-4690-964e-b577ced1d0bd" (UID: "e2b26284-2596-4690-964e-b577ced1d0bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:54.916956 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.916929    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util\") pod \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") "
Sep 11 03:46:54.917071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.916997    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zvgs\" (UniqueName: \"kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs\") pod \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") "
Sep 11 03:46:54.917071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917026    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle\") pod \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\" (UID: \"bf516d4c-ba08-4408-aec6-14b0ef49dfd8\") "
Sep 11 03:46:54.917279 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917262    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qth4x\" (UniqueName: \"kubernetes.io/projected/e2b26284-2596-4690-964e-b577ced1d0bd-kube-api-access-qth4x\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917286    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917298    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917310    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2b26284-2596-4690-964e-b577ced1d0bd-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917323    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ng8q\" (UniqueName: \"kubernetes.io/projected/010fdffc-044c-45f1-955a-d8bc022d0075-kube-api-access-4ng8q\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917337    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/010fdffc-044c-45f1-955a-d8bc022d0075-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:54.917555 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.917533    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle" (OuterVolumeSpecName: "bundle") pod "bf516d4c-ba08-4408-aec6-14b0ef49dfd8" (UID: "bf516d4c-ba08-4408-aec6-14b0ef49dfd8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:54.919160 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.919133    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs" (OuterVolumeSpecName: "kube-api-access-9zvgs") pod "bf516d4c-ba08-4408-aec6-14b0ef49dfd8" (UID: "bf516d4c-ba08-4408-aec6-14b0ef49dfd8"). InnerVolumeSpecName "kube-api-access-9zvgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:46:54.922374 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:54.922346    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util" (OuterVolumeSpecName: "util") pod "bf516d4c-ba08-4408-aec6-14b0ef49dfd8" (UID: "bf516d4c-ba08-4408-aec6-14b0ef49dfd8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:46:55.018541 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.018516    2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-util\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:55.018541 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.018538    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zvgs\" (UniqueName: \"kubernetes.io/projected/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-kube-api-access-9zvgs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:55.018707 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.018549    2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf516d4c-ba08-4408-aec6-14b0ef49dfd8-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:46:55.604587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.604555    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6"
Sep 11 03:46:55.604587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.604580    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apswp6" event={"ID":"e2b26284-2596-4690-964e-b577ced1d0bd","Type":"ContainerDied","Data":"5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe"}
Sep 11 03:46:55.605109 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.604625    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f641ef9d7d51a80af82b91bb5fc528e821552fc7994d2cfef0f23788e85ebbe"
Sep 11 03:46:55.606747 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.606710    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs" event={"ID":"010fdffc-044c-45f1-955a-d8bc022d0075","Type":"ContainerDied","Data":"094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d"}
Sep 11 03:46:55.606900 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.606749    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c69frhs"
Sep 11 03:46:55.606900 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.606752    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="094f955b97eacc6bc7b981243371898ec58711fc1ff01144af1126e3a6a1c25d"
Sep 11 03:46:55.609014 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.608995    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275"
Sep 11 03:46:55.609125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.608992    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpv275" event={"ID":"bf516d4c-ba08-4408-aec6-14b0ef49dfd8","Type":"ContainerDied","Data":"2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c"}
Sep 11 03:46:55.609125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:46:55.609106    2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fc830c708ce954cfcf50ffeddbd0c1d749b9b9ec05408c06ea2f24f92c0fd2c"
Sep 11 03:47:00.338029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:00.337994    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:02.382669 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.382635    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"]
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.382978    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.382993    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383004    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383009    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383016    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383022    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383029    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383034    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383042    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383048    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383057    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383062    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383067    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383072    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="util"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383078    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383083    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383088    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383093    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383101    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383105    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="pull"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383112    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383117    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383129    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="extract"
Sep 11 03:47:02.383152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383136    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="extract"
Sep 11 03:47:02.383859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383187    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="010fdffc-044c-45f1-955a-d8bc022d0075" containerName="extract"
Sep 11 03:47:02.383859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383195    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2b26284-2596-4690-964e-b577ced1d0bd" containerName="extract"
Sep 11 03:47:02.383859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383201    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="de078a10-9474-40a7-9ab7-cf72ba8a7993" containerName="extract"
Sep 11 03:47:02.383859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.383208    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf516d4c-ba08-4408-aec6-14b0ef49dfd8" containerName="extract"
Sep 11 03:47:02.387099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.387084    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:02.390388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.390358    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r6z9x\""
Sep 11 03:47:02.390516 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.390454    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 03:47:02.391851 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.391801    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 03:47:02.396530 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.396506    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"]
Sep 11 03:47:02.481169 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.481133    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bksmv\" (UniqueName: \"kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv\") pod \"limitador-operator-controller-manager-74495c69f8-5ft97\" (UID: \"b122b459-fc6a-4d6d-8a87-654cc1dae8fa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:02.582016 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.581985    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bksmv\" (UniqueName: \"kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv\") pod \"limitador-operator-controller-manager-74495c69f8-5ft97\" (UID: \"b122b459-fc6a-4d6d-8a87-654cc1dae8fa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:02.596301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.596272    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bksmv\" (UniqueName: \"kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv\") pod \"limitador-operator-controller-manager-74495c69f8-5ft97\" (UID: \"b122b459-fc6a-4d6d-8a87-654cc1dae8fa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:02.703029 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.702937    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:02.934878 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.934840    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:02.974695 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:02.974669    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"]
Sep 11 03:47:02.976254 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:02.976226    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb122b459_fc6a_4d6d_8a87_654cc1dae8fa.slice/crio-757cfe43d0efdf02036a51167b67bfd4c3745c22c0584152d3bf3a93aa0c0043 WatchSource:0}: Error finding container 757cfe43d0efdf02036a51167b67bfd4c3745c22c0584152d3bf3a93aa0c0043: Status 404 returned error can't find the container with id 757cfe43d0efdf02036a51167b67bfd4c3745c22c0584152d3bf3a93aa0c0043
Sep 11 03:47:03.333745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:03.333713    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:03.652520 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:03.652429    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97" event={"ID":"b122b459-fc6a-4d6d-8a87-654cc1dae8fa","Type":"ContainerStarted","Data":"757cfe43d0efdf02036a51167b67bfd4c3745c22c0584152d3bf3a93aa0c0043"}
Sep 11 03:47:03.730665 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:03.730607    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:04.243367 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:04.243295    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:05.665584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:05.665548    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97" event={"ID":"b122b459-fc6a-4d6d-8a87-654cc1dae8fa","Type":"ContainerStarted","Data":"37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527"}
Sep 11 03:47:05.665999 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:05.665717    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:05.685671 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:05.685625    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97" podStartSLOduration=1.404941103 podStartE2EDuration="3.685610241s" totalImagesPullingTime="2.280669138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:05.684975353 +0000 UTC m=+415.327765515" watchObservedRunningTime="2026-09-11 03:47:05.685610241 +0000 UTC m=+415.328400403"
Sep 11 03:47:06.479013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.478973    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"]
Sep 11 03:47:06.482273 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.482249    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.485264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.485241    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f82vk\""
Sep 11 03:47:06.497099 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.497075    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"]
Sep 11 03:47:06.611506 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.611466    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:06.620593 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.620570    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rchrq\" (UniqueName: \"kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.620713 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.620696    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.721243 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.721214    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.721243 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.721246    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rchrq\" (UniqueName: \"kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.721700 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.721538    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.729560 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.729498    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:06.730808 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.730785    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rchrq\" (UniqueName: \"kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:06.792483 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:06.792452    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:07.068078 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:07.068053    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"]
Sep 11 03:47:07.070287 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:07.070258    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d5cb37_6e13_4878_9474_c31a50435031.slice/crio-e4e873eb72ef43f1e8f6382d078cd657177587682bbe1092cd4723073dd67414 WatchSource:0}: Error finding container e4e873eb72ef43f1e8f6382d078cd657177587682bbe1092cd4723073dd67414: Status 404 returned error can't find the container with id e4e873eb72ef43f1e8f6382d078cd657177587682bbe1092cd4723073dd67414
Sep 11 03:47:07.333249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:07.333163    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:07.678446 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:07.678357    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q" event={"ID":"23d5cb37-6e13-4878-9474-c31a50435031","Type":"ContainerStarted","Data":"e4e873eb72ef43f1e8f6382d078cd657177587682bbe1092cd4723073dd67414"}
Sep 11 03:47:07.729683 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:07.729649    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:07.827301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:07.827271    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:09.204912 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.204876    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fdrgk"]
Sep 11 03:47:09.208318 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.208295    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:09.211055 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.211031    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cp7qx\""
Sep 11 03:47:09.222506 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.222476    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:09.222747 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.222721    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fdrgk"]
Sep 11 03:47:09.312389 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.312351    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:09.345441 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.345405    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c859t\" (UniqueName: \"kubernetes.io/projected/b166f2c4-85b9-43a4-96c3-eedb529961a3-kube-api-access-c859t\") pod \"authorino-operator-76b478bd78-fdrgk\" (UID: \"b166f2c4-85b9-43a4-96c3-eedb529961a3\") " pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:09.446877 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.446810    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c859t\" (UniqueName: \"kubernetes.io/projected/b166f2c4-85b9-43a4-96c3-eedb529961a3-kube-api-access-c859t\") pod \"authorino-operator-76b478bd78-fdrgk\" (UID: \"b166f2c4-85b9-43a4-96c3-eedb529961a3\") " pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:09.456010 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.455921    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c859t\" (UniqueName: \"kubernetes.io/projected/b166f2c4-85b9-43a4-96c3-eedb529961a3-kube-api-access-c859t\") pod \"authorino-operator-76b478bd78-fdrgk\" (UID: \"b166f2c4-85b9-43a4-96c3-eedb529961a3\") " pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:09.520987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.520954    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:09.802005 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.801972    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-fdrgk"]
Sep 11 03:47:09.804555 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:09.804525    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb166f2c4_85b9_43a4_96c3_eedb529961a3.slice/crio-0fc7f453f3311ba22fbbbd7f76fb543a63f35a5f32db7bda3b4f991d4e0dfbbb WatchSource:0}: Error finding container 0fc7f453f3311ba22fbbbd7f76fb543a63f35a5f32db7bda3b4f991d4e0dfbbb: Status 404 returned error can't find the container with id 0fc7f453f3311ba22fbbbd7f76fb543a63f35a5f32db7bda3b4f991d4e0dfbbb
Sep 11 03:47:09.953277 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:09.953219    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:10.631205 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:10.631167    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:10.697881 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:10.697836    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk" event={"ID":"b166f2c4-85b9-43a4-96c3-eedb529961a3","Type":"ContainerStarted","Data":"0fc7f453f3311ba22fbbbd7f76fb543a63f35a5f32db7bda3b4f991d4e0dfbbb"}
Sep 11 03:47:11.706103 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:11.706069    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q" event={"ID":"23d5cb37-6e13-4878-9474-c31a50435031","Type":"ContainerStarted","Data":"26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862"}
Sep 11 03:47:11.706553 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:11.706214    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:11.729258 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:11.729205    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q" podStartSLOduration=1.636829816 podStartE2EDuration="5.729190882s" totalImagesPullingTime="4.092361066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:11.727110655 +0000 UTC m=+421.369900830" watchObservedRunningTime="2026-09-11 03:47:11.729190882 +0000 UTC m=+421.371981043"
Sep 11 03:47:12.712568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:12.712522    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk" event={"ID":"b166f2c4-85b9-43a4-96c3-eedb529961a3","Type":"ContainerStarted","Data":"65903e5b35866f10adbce6fcb55f8af64214c95ca1b8e7d9fee3937d680bf2dc"}
Sep 11 03:47:12.713013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:12.712661    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:12.733039 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:12.732994    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk" podStartSLOduration=1.6901973689999998 podStartE2EDuration="3.732979625s" totalImagesPullingTime="2.042782256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:12.73183169 +0000 UTC m=+422.374621849" watchObservedRunningTime="2026-09-11 03:47:12.732979625 +0000 UTC m=+422.375769786"
Sep 11 03:47:12.965419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:12.965334    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:13.097476 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.097436    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:13.797690 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.797653    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:13.801960 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.801935    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"]
Sep 11 03:47:13.805700 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.805681    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.808287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.808259    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 03:47:13.808287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.808279    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 03:47:13.808442 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.808394    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t5r8x\""
Sep 11 03:47:13.813575 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.813549    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"]
Sep 11 03:47:13.887746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.887711    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6114465-febc-4f35-83ec-a8371655abc5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.887746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.887751    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r9h5\" (UniqueName: \"kubernetes.io/projected/f6114465-febc-4f35-83ec-a8371655abc5-kube-api-access-6r9h5\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.888012 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.887848    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.988883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.988847    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6114465-febc-4f35-83ec-a8371655abc5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.988883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.988888    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r9h5\" (UniqueName: \"kubernetes.io/projected/f6114465-febc-4f35-83ec-a8371655abc5-kube-api-access-6r9h5\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.989090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.988932    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.989090 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:13.989065    2553 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 11 03:47:13.989157 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:13.989139    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert podName:f6114465-febc-4f35-83ec-a8371655abc5 nodeName:}" failed. No retries permitted until 2026-09-11 03:47:14.489119681 +0000 UTC m=+424.131909824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-5wbrq" (UID: "f6114465-febc-4f35-83ec-a8371655abc5") : secret "plugin-serving-cert" not found
Sep 11 03:47:13.989478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.989460    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f6114465-febc-4f35-83ec-a8371655abc5-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:13.997985 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:13.997958    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r9h5\" (UniqueName: \"kubernetes.io/projected/f6114465-febc-4f35-83ec-a8371655abc5-kube-api-access-6r9h5\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:14.043851 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.043799    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:14.443286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.443245    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:14.492316 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.492282    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:14.494688 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.494665    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6114465-febc-4f35-83ec-a8371655abc5-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-5wbrq\" (UID: \"f6114465-febc-4f35-83ec-a8371655abc5\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:14.503494 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.503469    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:14.715475 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.715390    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"
Sep 11 03:47:14.983193 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:14.983169    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq"]
Sep 11 03:47:14.984868 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:14.984840    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6114465_febc_4f35_83ec_a8371655abc5.slice/crio-812a609f122eef6df2570a8a89bfa835e11835a087f124be04d09af3f151824e WatchSource:0}: Error finding container 812a609f122eef6df2570a8a89bfa835e11835a087f124be04d09af3f151824e: Status 404 returned error can't find the container with id 812a609f122eef6df2570a8a89bfa835e11835a087f124be04d09af3f151824e
Sep 11 03:47:15.731883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:15.731842    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq" event={"ID":"f6114465-febc-4f35-83ec-a8371655abc5","Type":"ContainerStarted","Data":"812a609f122eef6df2570a8a89bfa835e11835a087f124be04d09af3f151824e"}
Sep 11 03:47:16.673999 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:16.673967    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:16.746741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:16.746707    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:16.799436 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:16.799401    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:17.179403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:17.179364    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:17.277715 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:17.277674    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:17.570358 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:17.570316    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:17.692991 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:17.692954    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:19.760224 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:19.760118    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq" event={"ID":"f6114465-febc-4f35-83ec-a8371655abc5","Type":"ContainerStarted","Data":"fb0a64866a910bc23be8a88762b933102d31ebcf1ba670c7b94d5228ac2b9844"}
Sep 11 03:47:19.775428 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:19.775380    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-5wbrq" podStartSLOduration=2.361013818 podStartE2EDuration="6.775362194s" totalImagesPullingTime="4.414348376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:19.774474159 +0000 UTC m=+429.417264335" watchObservedRunningTime="2026-09-11 03:47:19.775362194 +0000 UTC m=+429.418152357"
Sep 11 03:47:22.714757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:22.714727    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:23.720505 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:23.720461    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-fdrgk"
Sep 11 03:47:24.926054 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.926021    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"]
Sep 11 03:47:24.926434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.926203    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q" podUID="23d5cb37-6e13-4878-9474-c31a50435031" containerName="manager" containerID="cri-o://26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862" gracePeriod=2
Sep 11 03:47:24.938895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.938860    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"]
Sep 11 03:47:24.946144 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:24.946103    2553 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"23d5cb37-6e13-4878-9474-c31a50435031\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\": pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:24.954035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.954011    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"]
Sep 11 03:47:24.954257 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.954233    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" containerName="manager" containerID="cri-o://37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527" gracePeriod=2
Sep 11 03:47:24.956286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.956262    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:24.956620 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.956596    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d5cb37-6e13-4878-9474-c31a50435031" containerName="manager"
Sep 11 03:47:24.956620 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.956612    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d5cb37-6e13-4878-9474-c31a50435031" containerName="manager"
Sep 11 03:47:24.956802 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.956698    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d5cb37-6e13-4878-9474-c31a50435031" containerName="manager"
Sep 11 03:47:24.959033 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:24.958995    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:24.960123 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.960104    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:24.962429 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:24.962389    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:24.964080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.964052    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"]
Sep 11 03:47:24.967293 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:24.967270    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:24.974777 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:24.974747    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:24.976360 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.976338    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:24.982068 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.982044    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"]
Sep 11 03:47:24.982381 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.982365    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" containerName="manager"
Sep 11 03:47:24.982452 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.982385    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" containerName="manager"
Sep 11 03:47:24.982547 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.982534    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" containerName="manager"
Sep 11 03:47:24.985539 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.985506    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:24.997187 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:24.997165    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"]
Sep 11 03:47:25.010163 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.010128    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.012279 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.012252    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.083695 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.083668    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwxh\" (UniqueName: \"kubernetes.io/projected/077a4a5d-e14e-4118-93e2-57d281e98b5b-kube-api-access-cvwxh\") pod \"limitador-operator-controller-manager-74495c69f8-blf9s\" (UID: \"077a4a5d-e14e-4118-93e2-57d281e98b5b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:25.083810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.083730    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.083810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.083760    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgn96\" (UniqueName: \"kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.184564 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.184531    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwxh\" (UniqueName: \"kubernetes.io/projected/077a4a5d-e14e-4118-93e2-57d281e98b5b-kube-api-access-cvwxh\") pod \"limitador-operator-controller-manager-74495c69f8-blf9s\" (UID: \"077a4a5d-e14e-4118-93e2-57d281e98b5b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:25.184674 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.184627    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.184674 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.184659    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgn96\" (UniqueName: \"kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.185048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.185022    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.189212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.189187    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.191478 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.191436    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.193467 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.193448    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgn96\" (UniqueName: \"kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4fpw2\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.193467 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.193460    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.193606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.193471    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwxh\" (UniqueName: \"kubernetes.io/projected/077a4a5d-e14e-4118-93e2-57d281e98b5b-kube-api-access-cvwxh\") pod \"limitador-operator-controller-manager-74495c69f8-blf9s\" (UID: \"077a4a5d-e14e-4118-93e2-57d281e98b5b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:25.207888 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.207867    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.210350 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.210326    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.212364 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.212342    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.285857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.285805    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume\") pod \"23d5cb37-6e13-4878-9474-c31a50435031\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") "
Sep 11 03:47:25.285968 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.285899    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rchrq\" (UniqueName: \"kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq\") pod \"23d5cb37-6e13-4878-9474-c31a50435031\" (UID: \"23d5cb37-6e13-4878-9474-c31a50435031\") "
Sep 11 03:47:25.286261 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.286238    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "23d5cb37-6e13-4878-9474-c31a50435031" (UID: "23d5cb37-6e13-4878-9474-c31a50435031"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:47:25.288117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.288094    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq" (OuterVolumeSpecName: "kube-api-access-rchrq") pod "23d5cb37-6e13-4878-9474-c31a50435031" (UID: "23d5cb37-6e13-4878-9474-c31a50435031"). InnerVolumeSpecName "kube-api-access-rchrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:47:25.376478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.376446    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.385876 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.385846    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:25.387085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.387060    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bksmv\" (UniqueName: \"kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv\") pod \"b122b459-fc6a-4d6d-8a87-654cc1dae8fa\" (UID: \"b122b459-fc6a-4d6d-8a87-654cc1dae8fa\") "
Sep 11 03:47:25.387245 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.387223    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:47:25.387288 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.387257    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rchrq\" (UniqueName: \"kubernetes.io/projected/23d5cb37-6e13-4878-9474-c31a50435031-kube-api-access-rchrq\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:25.387288 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.387270    2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/23d5cb37-6e13-4878-9474-c31a50435031-extensions-socket-volume\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:25.389202 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.389174    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv" (OuterVolumeSpecName: "kube-api-access-bksmv") pod "b122b459-fc6a-4d6d-8a87-654cc1dae8fa" (UID: "b122b459-fc6a-4d6d-8a87-654cc1dae8fa"). InnerVolumeSpecName "kube-api-access-bksmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:47:25.389988 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.389956    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.392147 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.392111    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.443049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.442492    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:25.487807 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.487772    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bksmv\" (UniqueName: \"kubernetes.io/projected/b122b459-fc6a-4d6d-8a87-654cc1dae8fa-kube-api-access-bksmv\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:25.541263 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.541225    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:25.671038 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.671007    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:25.674745 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:25.674720    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a9550b_6eeb_4af6_9b8a_c6a12eff9f06.slice/crio-074394f17edecc9895a8b097f3544ed246d001b290263eb96889093d50cd0a65 WatchSource:0}: Error finding container 074394f17edecc9895a8b097f3544ed246d001b290263eb96889093d50cd0a65: Status 404 returned error can't find the container with id 074394f17edecc9895a8b097f3544ed246d001b290263eb96889093d50cd0a65
Sep 11 03:47:25.678047 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.678027    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"]
Sep 11 03:47:25.678771 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:25.678735    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077a4a5d_e14e_4118_93e2_57d281e98b5b.slice/crio-7c430a921076cb6b6a54250c5844369936662772e02d7cf1a24a8649007cca01 WatchSource:0}: Error finding container 7c430a921076cb6b6a54250c5844369936662772e02d7cf1a24a8649007cca01: Status 404 returned error can't find the container with id 7c430a921076cb6b6a54250c5844369936662772e02d7cf1a24a8649007cca01
Sep 11 03:47:25.693326 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.693305    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:25.778186 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.778156    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:25.796660 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.796629    2553 generic.go:356] "Generic (PLEG): container finished" podID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" containerID="37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527" exitCode=0
Sep 11 03:47:25.796799 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.796677    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.796799 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.796730    2553 scope.go:117] "RemoveContainer" containerID="37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527"
Sep 11 03:47:25.803459 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.799829    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.803459 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.802942    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.805124 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.805099    2553 generic.go:356] "Generic (PLEG): container finished" podID="23d5cb37-6e13-4878-9474-c31a50435031" containerID="26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862" exitCode=0
Sep 11 03:47:25.805239 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.805178    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.807321 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.807297    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" event={"ID":"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06","Type":"ContainerStarted","Data":"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"}
Sep 11 03:47:25.807516 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.807378    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" event={"ID":"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06","Type":"ContainerStarted","Data":"074394f17edecc9895a8b097f3544ed246d001b290263eb96889093d50cd0a65"}
Sep 11 03:47:25.807516 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.807438    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:25.808234 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.808060    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.809973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.809928    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s" event={"ID":"077a4a5d-e14e-4118-93e2-57d281e98b5b","Type":"ContainerStarted","Data":"ee685c5a46f6e33a0e9864a5387586637ba70388683e98d1463cb830d2e25986"}
Sep 11 03:47:25.809973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.809973    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s" event={"ID":"077a4a5d-e14e-4118-93e2-57d281e98b5b","Type":"ContainerStarted","Data":"7c430a921076cb6b6a54250c5844369936662772e02d7cf1a24a8649007cca01"}
Sep 11 03:47:25.810158 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.810138    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:25.810224 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.810175    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.812343 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.812311    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.814316 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.814203    2553 scope.go:117] "RemoveContainer" containerID="37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527"
Sep 11 03:47:25.814403 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.814376    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:25.814614 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.814572    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527\": container with ID starting with 37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527 not found: ID does not exist" containerID="37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527"
Sep 11 03:47:25.814614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.814614    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527"} err="failed to get container status \"37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527\": rpc error: code = NotFound desc = could not find container \"37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527\": container with ID starting with 37747f762493e38c99442dc3f29697f6eb5b28bb974b4b50ee28e790186df527 not found: ID does not exist"
Sep 11 03:47:25.814737 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.814632    2553 scope.go:117] "RemoveContainer" containerID="26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862"
Sep 11 03:47:25.822740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.822725    2553 scope.go:117] "RemoveContainer" containerID="26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862"
Sep 11 03:47:25.823010 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.822993    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862\": container with ID starting with 26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862 not found: ID does not exist" containerID="26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862"
Sep 11 03:47:25.823096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.823012    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862"} err="failed to get container status \"26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862\": rpc error: code = NotFound desc = could not find container \"26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862\": container with ID starting with 26111f1c20ef10ea1e8f09e1c44c746353346292c0a79a8cb5166b80340ce862 not found: ID does not exist"
Sep 11 03:47:25.834351 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.834313    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s" podStartSLOduration=1.834301986 podStartE2EDuration="1.834301986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:25.833852857 +0000 UTC m=+435.476643010" watchObservedRunningTime="2026-09-11 03:47:25.834301986 +0000 UTC m=+435.477092147"
Sep 11 03:47:25.856356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:25.856314    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" podStartSLOduration=1.856301086 podStartE2EDuration="1.856301086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:25.856121424 +0000 UTC m=+435.498911587" watchObservedRunningTime="2026-09-11 03:47:25.856301086 +0000 UTC m=+435.499091246"
Sep 11 03:47:25.858325 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.858293    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gwn5q\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="23d5cb37-6e13-4878-9474-c31a50435031" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gwn5q"
Sep 11 03:47:25.860319 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:25.860292    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-5ft97\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5ft97"
Sep 11 03:47:26.872938 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:26.872904    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d5cb37-6e13-4878-9474-c31a50435031" path="/var/lib/kubelet/pods/23d5cb37-6e13-4878-9474-c31a50435031/volumes"
Sep 11 03:47:26.873328 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:26.873230    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b122b459-fc6a-4d6d-8a87-654cc1dae8fa" path="/var/lib/kubelet/pods/b122b459-fc6a-4d6d-8a87-654cc1dae8fa/volumes"
Sep 11 03:47:27.839780 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:27.839744    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:27.942178 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:27.942134    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:32.260505 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:32.260454    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:32.360982 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:32.360943    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:35.986731 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:35.986644    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:36.088409 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:36.088372    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:36.187922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:36.187888    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:36.288589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:36.288549    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:36.820046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:36.820001    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:36.820212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:36.820087    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-blf9s"
Sep 11 03:47:37.195390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:37.195291    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:37.293840 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:37.293783    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:39.929190 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:39.929156    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:39.929617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:39.929338    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" podUID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" containerName="manager" containerID="cri-o://f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2" gracePeriod=10
Sep 11 03:47:40.177227 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.177201    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:40.201085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.201016    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume\") pod \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") "
Sep 11 03:47:40.201085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.201069    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgn96\" (UniqueName: \"kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96\") pod \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\" (UID: \"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06\") "
Sep 11 03:47:40.201413 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.201391    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" (UID: "91a9550b-6eeb-4af6-9b8a-c6a12eff9f06"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:47:40.203323 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.203272    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96" (OuterVolumeSpecName: "kube-api-access-vgn96") pod "91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" (UID: "91a9550b-6eeb-4af6-9b8a-c6a12eff9f06"). InnerVolumeSpecName "kube-api-access-vgn96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:47:40.301737 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.301697    2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-extensions-socket-volume\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:40.301737 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.301733    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgn96\" (UniqueName: \"kubernetes.io/projected/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06-kube-api-access-vgn96\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:40.892671 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.892641    2553 generic.go:356] "Generic (PLEG): container finished" podID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" containerID="f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2" exitCode=0
Sep 11 03:47:40.892835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.892701    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" event={"ID":"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06","Type":"ContainerDied","Data":"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"}
Sep 11 03:47:40.892835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.892712    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"
Sep 11 03:47:40.892835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.892740    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2" event={"ID":"91a9550b-6eeb-4af6-9b8a-c6a12eff9f06","Type":"ContainerDied","Data":"074394f17edecc9895a8b097f3544ed246d001b290263eb96889093d50cd0a65"}
Sep 11 03:47:40.892835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.892760    2553 scope.go:117] "RemoveContainer" containerID="f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"
Sep 11 03:47:40.913723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.913701    2553 scope.go:117] "RemoveContainer" containerID="f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"
Sep 11 03:47:40.914064 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:40.914043    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2\": container with ID starting with f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2 not found: ID does not exist" containerID="f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"
Sep 11 03:47:40.914130 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.914072    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2"} err="failed to get container status \"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2\": rpc error: code = NotFound desc = could not find container \"f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2\": container with ID starting with f1984afce6e28205d2943f17ba7f072f6a7446c0bdf9b15ef28cefd2585fd5e2 not found: ID does not exist"
Sep 11 03:47:40.917954 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.917933    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:40.925182 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:40.925164    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4fpw2"]
Sep 11 03:47:42.872741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:42.872710    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" path="/var/lib/kubelet/pods/91a9550b-6eeb-4af6-9b8a-c6a12eff9f06/volumes"
Sep 11 03:47:46.390246 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:46.390213    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:46.490241 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:46.490197    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:48.829459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:48.829423    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:48.886975 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:48.886948    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:50.409935 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.409895    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76fbf4865d-q7f9v" podUID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" containerName="console" containerID="cri-o://59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7" gracePeriod=15
Sep 11 03:47:50.647310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.647283    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:47:50.680860 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680783    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.680860 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680837    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680860    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680904    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680925    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680965    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8grh\" (UniqueName: \"kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.680995    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config\") pod \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\" (UID: \"35e7fdfd-b431-4b76-9477-6a894fa35fe6\") "
Sep 11 03:47:50.681317 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.681284    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config" (OuterVolumeSpecName: "console-config") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:47:50.681317 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.681304    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca" (OuterVolumeSpecName: "service-ca") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:47:50.681440 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.681314    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:47:50.681440 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.681399    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:47:50.682949 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.682924    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:47:50.683112 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.683088    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh" (OuterVolumeSpecName: "kube-api-access-f8grh") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "kube-api-access-f8grh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:47:50.683228 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.683210    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "35e7fdfd-b431-4b76-9477-6a894fa35fe6" (UID: "35e7fdfd-b431-4b76-9477-6a894fa35fe6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:47:50.782443 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782418    2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-config\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782443 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782442    2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782451    2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-serving-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782460    2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-trusted-ca-bundle\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782470    2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35e7fdfd-b431-4b76-9477-6a894fa35fe6-oauth-serving-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782480    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8grh\" (UniqueName: \"kubernetes.io/projected/35e7fdfd-b431-4b76-9477-6a894fa35fe6-kube-api-access-f8grh\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.782622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.782488    2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35e7fdfd-b431-4b76-9477-6a894fa35fe6-console-oauth-config\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:47:50.943750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.943658    2553 generic.go:356] "Generic (PLEG): container finished" podID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" containerID="59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7" exitCode=0
Sep 11 03:47:50.943750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.943711    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76fbf4865d-q7f9v"
Sep 11 03:47:50.943750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.943725    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76fbf4865d-q7f9v" event={"ID":"35e7fdfd-b431-4b76-9477-6a894fa35fe6","Type":"ContainerDied","Data":"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"}
Sep 11 03:47:50.944037 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.943771    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76fbf4865d-q7f9v" event={"ID":"35e7fdfd-b431-4b76-9477-6a894fa35fe6","Type":"ContainerDied","Data":"ad569e00dde7eaf76baee54d4e4cfe3cea3ba9ffa74bc09b47c1f10e2f931ed0"}
Sep 11 03:47:50.944037 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.943797    2553 scope.go:117] "RemoveContainer" containerID="59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"
Sep 11 03:47:50.953317 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.953299    2553 scope.go:117] "RemoveContainer" containerID="59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"
Sep 11 03:47:50.953574 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:47:50.953557    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7\": container with ID starting with 59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7 not found: ID does not exist" containerID="59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"
Sep 11 03:47:50.953621 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.953582    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7"} err="failed to get container status \"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7\": rpc error: code = NotFound desc = could not find container \"59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7\": container with ID starting with 59161808b02811a35e26ddf0a91316a6069c38dc25fec761993c14a216ec2ef7 not found: ID does not exist"
Sep 11 03:47:50.960979 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.960958    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:47:50.964580 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:50.964562    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76fbf4865d-q7f9v"]
Sep 11 03:47:52.871546 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:52.871503    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" path="/var/lib/kubelet/pods/35e7fdfd-b431-4b76-9477-6a894fa35fe6/volumes"
Sep 11 03:47:56.641483 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641451    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"]
Sep 11 03:47:56.641953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641795    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" containerName="console"
Sep 11 03:47:56.641953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641807    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" containerName="console"
Sep 11 03:47:56.641953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641840    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" containerName="manager"
Sep 11 03:47:56.641953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641851    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" containerName="manager"
Sep 11 03:47:56.642211 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.641973    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="91a9550b-6eeb-4af6-9b8a-c6a12eff9f06" containerName="manager"
Sep 11 03:47:56.642211 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.642020    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e7fdfd-b431-4b76-9477-6a894fa35fe6" containerName="console"
Sep 11 03:47:56.646390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.646369    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.649007 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.648986    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dvn4g\""
Sep 11 03:47:56.657981 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.657961    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"]
Sep 11 03:47:56.672400 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.672363    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:56.725524 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725496    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725639 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725533    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725639 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725568    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725639 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725627    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppzf4\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-kube-api-access-ppzf4\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725659    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725694    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725713    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725962 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725852    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.725962 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.725882    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.826961 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.826926    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.826972    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppzf4\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-kube-api-access-ppzf4\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827014    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827058    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827180    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827272    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827304    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827329    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827342    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827385    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827383    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827432    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827649 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827516    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.827992 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.827967    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.829674 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.829647    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.829770 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.829699    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.839091 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.839068    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppzf4\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-kube-api-access-ppzf4\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.839194 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.839069    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d9335c5-6a0b-4020-98de-d4eaafdfba1b-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-5zqtx\" (UID: \"6d9335c5-6a0b-4020-98de-d4eaafdfba1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:56.849394 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.849368    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:47:56.957335 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:56.957247    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:57.231021 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.230991    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"]
Sep 11 03:47:57.232258 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:47:57.232230    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9335c5_6a0b_4020_98de_d4eaafdfba1b.slice/crio-baef7316d874f7c6b062102db256589719ebda25aa04b4e776499ab8b2b4080f WatchSource:0}: Error finding container baef7316d874f7c6b062102db256589719ebda25aa04b4e776499ab8b2b4080f: Status 404 returned error can't find the container with id baef7316d874f7c6b062102db256589719ebda25aa04b4e776499ab8b2b4080f
Sep 11 03:47:57.234258 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.234222    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:47:57.234336 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.234287    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:47:57.234336 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.234318    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:47:57.983096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.983064    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" event={"ID":"6d9335c5-6a0b-4020-98de-d4eaafdfba1b","Type":"ContainerStarted","Data":"39f692d5b247e8132a293682493faa0c882ac2f57f3f8a4844324867bc73eb94"}
Sep 11 03:47:57.983486 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:57.983114    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" event={"ID":"6d9335c5-6a0b-4020-98de-d4eaafdfba1b","Type":"ContainerStarted","Data":"baef7316d874f7c6b062102db256589719ebda25aa04b4e776499ab8b2b4080f"}
Sep 11 03:47:58.023333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:58.023283    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podStartSLOduration=2.023267925 podStartE2EDuration="2.023267925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:47:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:47:58.020002852 +0000 UTC m=+467.662793013" watchObservedRunningTime="2026-09-11 03:47:58.023267925 +0000 UTC m=+467.666058087"
Sep 11 03:47:58.957366 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:58.957318    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:58.961839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:58.961805    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:58.989049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:58.989029    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:47:58.989768 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:47:58.989752    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:48:13.996057 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:13.996023    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:13.999570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:13.999539    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:14.000926 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.000905    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.003692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.003669    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 03:48:14.005803 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.005783    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:14.068212 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.068147    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.068422 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.068283    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlvh\" (UniqueName: \"kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.094554 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.094520    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:14.104014 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.103980    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:14.169425 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.169389    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlvh\" (UniqueName: \"kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.169614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.169440    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.170075 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.170056    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.177872 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.177841    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlvh\" (UniqueName: \"kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh\") pod \"limitador-limitador-8588749977-4kx4x\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") " pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.206277 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.206248    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:14.311450 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.311416    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:14.581140 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.581115    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:14.582793 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:48:14.582765    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f9b416_abc6_4352_a327_735ff628f447.slice/crio-5ed45a8a2cf8ff99355c70fa4d4089832d828f546b00057f848dbfa25c6df92b WatchSource:0}: Error finding container 5ed45a8a2cf8ff99355c70fa4d4089832d828f546b00057f848dbfa25c6df92b: Status 404 returned error can't find the container with id 5ed45a8a2cf8ff99355c70fa4d4089832d828f546b00057f848dbfa25c6df92b
Sep 11 03:48:14.813247 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.813204    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:14.884628 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.884550    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:14.895223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.895202    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:14.899678 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.899657    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:14.902352 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.902326    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cjkt4\""
Sep 11 03:48:14.903422 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.903403    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:14.977205 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:14.977175    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxwph\" (UniqueName: \"kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph\") pod \"authorino-59c974bd48-khr67\" (UID: \"c8f495f1-815d-41d5-b981-cf98ddc6a56a\") " pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:15.070253 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:15.070222    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" event={"ID":"f0f9b416-abc6-4352-a327-735ff628f447","Type":"ContainerStarted","Data":"5ed45a8a2cf8ff99355c70fa4d4089832d828f546b00057f848dbfa25c6df92b"}
Sep 11 03:48:15.078570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:15.078539    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxwph\" (UniqueName: \"kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph\") pod \"authorino-59c974bd48-khr67\" (UID: \"c8f495f1-815d-41d5-b981-cf98ddc6a56a\") " pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:15.086539 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:15.086521    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxwph\" (UniqueName: \"kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph\") pod \"authorino-59c974bd48-khr67\" (UID: \"c8f495f1-815d-41d5-b981-cf98ddc6a56a\") " pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:15.209267 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:15.209188    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:15.477695 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:15.477665    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:15.614468 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:48:15.614430    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8f495f1_815d_41d5_b981_cf98ddc6a56a.slice/crio-1a7e17d32c46fece6c50563b21d7a55baab56853c04e06a4019195da99bdbd9d WatchSource:0}: Error finding container 1a7e17d32c46fece6c50563b21d7a55baab56853c04e06a4019195da99bdbd9d: Status 404 returned error can't find the container with id 1a7e17d32c46fece6c50563b21d7a55baab56853c04e06a4019195da99bdbd9d
Sep 11 03:48:16.076296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:16.076260    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-khr67" event={"ID":"c8f495f1-815d-41d5-b981-cf98ddc6a56a","Type":"ContainerStarted","Data":"1a7e17d32c46fece6c50563b21d7a55baab56853c04e06a4019195da99bdbd9d"}
Sep 11 03:48:16.077885 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:16.077860    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" event={"ID":"f0f9b416-abc6-4352-a327-735ff628f447","Type":"ContainerStarted","Data":"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"}
Sep 11 03:48:16.078032 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:16.078019    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:16.094608 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:16.094559    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" podStartSLOduration=2.007122676 podStartE2EDuration="3.094543145s" totalImagesPullingTime="1.087420469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:48:16.094065618 +0000 UTC m=+485.736855780" watchObservedRunningTime="2026-09-11 03:48:16.094543145 +0000 UTC m=+485.737333308"
Sep 11 03:48:18.088765 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:18.088728    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-khr67" event={"ID":"c8f495f1-815d-41d5-b981-cf98ddc6a56a","Type":"ContainerStarted","Data":"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"}
Sep 11 03:48:18.104293 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:18.104242    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-khr67" podStartSLOduration=2.65603673 podStartE2EDuration="4.10422586s" totalImagesPullingTime="1.44818913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:48:18.102476174 +0000 UTC m=+487.745266337" watchObservedRunningTime="2026-09-11 03:48:18.10422586 +0000 UTC m=+487.747016021"
Sep 11 03:48:26.740974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:26.740936    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:26.790045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:26.790015    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:27.084123 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:27.084095    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:27.137666 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:27.137615    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:27.137863 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:27.137799    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" podUID="f0f9b416-abc6-4352-a327-735ff628f447" containerName="limitador" containerID="cri-o://1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93" gracePeriod=30
Sep 11 03:48:28.079149 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.079124    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:28.141551 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.141519    2553 generic.go:356] "Generic (PLEG): container finished" podID="f0f9b416-abc6-4352-a327-735ff628f447" containerID="1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93" exitCode=0
Sep 11 03:48:28.141746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.141568    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x"
Sep 11 03:48:28.141746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.141603    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" event={"ID":"f0f9b416-abc6-4352-a327-735ff628f447","Type":"ContainerDied","Data":"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"}
Sep 11 03:48:28.141746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.141644    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4kx4x" event={"ID":"f0f9b416-abc6-4352-a327-735ff628f447","Type":"ContainerDied","Data":"5ed45a8a2cf8ff99355c70fa4d4089832d828f546b00057f848dbfa25c6df92b"}
Sep 11 03:48:28.141746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.141666    2553 scope.go:117] "RemoveContainer" containerID="1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"
Sep 11 03:48:28.150049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.150030    2553 scope.go:117] "RemoveContainer" containerID="1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"
Sep 11 03:48:28.150306 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:28.150287    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93\": container with ID starting with 1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93 not found: ID does not exist" containerID="1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"
Sep 11 03:48:28.150353 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.150315    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93"} err="failed to get container status \"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93\": rpc error: code = NotFound desc = could not find container \"1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93\": container with ID starting with 1b64ccc396f1a28d4cbd001e56b455e7cd14d62a67cca03ca9d89fbe16225e93 not found: ID does not exist"
Sep 11 03:48:28.184085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.184058    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmlvh\" (UniqueName: \"kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh\") pod \"f0f9b416-abc6-4352-a327-735ff628f447\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") "
Sep 11 03:48:28.184216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.184113    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file\") pod \"f0f9b416-abc6-4352-a327-735ff628f447\" (UID: \"f0f9b416-abc6-4352-a327-735ff628f447\") "
Sep 11 03:48:28.184502 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.184465    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file" (OuterVolumeSpecName: "config-file") pod "f0f9b416-abc6-4352-a327-735ff628f447" (UID: "f0f9b416-abc6-4352-a327-735ff628f447"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:48:28.186150 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.186121    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh" (OuterVolumeSpecName: "kube-api-access-cmlvh") pod "f0f9b416-abc6-4352-a327-735ff628f447" (UID: "f0f9b416-abc6-4352-a327-735ff628f447"). InnerVolumeSpecName "kube-api-access-cmlvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:48:28.285240 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.285207    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmlvh\" (UniqueName: \"kubernetes.io/projected/f0f9b416-abc6-4352-a327-735ff628f447-kube-api-access-cmlvh\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:48:28.285240 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.285237    2553 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f0f9b416-abc6-4352-a327-735ff628f447-config-file\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:48:28.463738 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.463709    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:28.467375 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.467350    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4kx4x"]
Sep 11 03:48:28.872704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:28.872670    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0f9b416-abc6-4352-a327-735ff628f447" path="/var/lib/kubelet/pods/f0f9b416-abc6-4352-a327-735ff628f447/volumes"
Sep 11 03:48:30.535912 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:30.535875    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:30.640880 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:30.640836    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:31.006608 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.006522    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"]
Sep 11 03:48:31.006884 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.006871    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0f9b416-abc6-4352-a327-735ff628f447" containerName="limitador"
Sep 11 03:48:31.006943 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.006885    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0f9b416-abc6-4352-a327-735ff628f447" containerName="limitador"
Sep 11 03:48:31.006980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.006947    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0f9b416-abc6-4352-a327-735ff628f447" containerName="limitador"
Sep 11 03:48:31.011259 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.011238    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.014041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.014014    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 03:48:31.014155 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.014081    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 03:48:31.015145 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.015126    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-b58bg\""
Sep 11 03:48:31.015234 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.015131    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 03:48:31.019744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.019722    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"]
Sep 11 03:48:31.040693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.040661    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:31.107614 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.107575    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/50c51cf4-16fd-47f2-8252-2714cfdef8e8-data\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.107763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.107670    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt9rl\" (UniqueName: \"kubernetes.io/projected/50c51cf4-16fd-47f2-8252-2714cfdef8e8-kube-api-access-lt9rl\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.209157 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.209122    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/50c51cf4-16fd-47f2-8252-2714cfdef8e8-data\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.209328 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.209174    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt9rl\" (UniqueName: \"kubernetes.io/projected/50c51cf4-16fd-47f2-8252-2714cfdef8e8-kube-api-access-lt9rl\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.209496 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.209477    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/50c51cf4-16fd-47f2-8252-2714cfdef8e8-data\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.218720 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.218690    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt9rl\" (UniqueName: \"kubernetes.io/projected/50c51cf4-16fd-47f2-8252-2714cfdef8e8-kube-api-access-lt9rl\") pod \"postgres-69fb4594cf-xn8b2\" (UID: \"50c51cf4-16fd-47f2-8252-2714cfdef8e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.322672 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.322644    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:31.591424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:31.591395    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"]
Sep 11 03:48:31.593046 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:48:31.593021    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c51cf4_16fd_47f2_8252_2714cfdef8e8.slice/crio-4d7ab783f9a989b75335d0d30747e13fde33ecf71d5e234e30d859b2e656b75d WatchSource:0}: Error finding container 4d7ab783f9a989b75335d0d30747e13fde33ecf71d5e234e30d859b2e656b75d: Status 404 returned error can't find the container with id 4d7ab783f9a989b75335d0d30747e13fde33ecf71d5e234e30d859b2e656b75d
Sep 11 03:48:32.164602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:32.164565    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2" event={"ID":"50c51cf4-16fd-47f2-8252-2714cfdef8e8","Type":"ContainerStarted","Data":"4d7ab783f9a989b75335d0d30747e13fde33ecf71d5e234e30d859b2e656b75d"}
Sep 11 03:48:34.177874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:34.177767    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2" event={"ID":"50c51cf4-16fd-47f2-8252-2714cfdef8e8","Type":"ContainerStarted","Data":"0e8ea1b1b0bd24f5ea5ce4fbeac031b7525f934b839e663459e2cf0ab4332145"}
Sep 11 03:48:34.177874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:34.177858    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:34.194931 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:34.194890    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2" podStartSLOduration=1.9547550089999999 podStartE2EDuration="4.194876149s" totalImagesPullingTime="2.24012114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:48:34.194632362 +0000 UTC m=+503.837422526" watchObservedRunningTime="2026-09-11 03:48:34.194876149 +0000 UTC m=+503.837666310"
Sep 11 03:48:40.211789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:40.211752    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xn8b2"
Sep 11 03:48:40.516687 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:40.516655    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:40.633764 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:40.633725    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:42.248306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.248274    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:42.248893 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.248547    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-khr67" podUID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" containerName="authorino" containerID="cri-o://e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28" gracePeriod=30
Sep 11 03:48:42.507406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.507337    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:42.597252 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.597215    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxwph\" (UniqueName: \"kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph\") pod \"c8f495f1-815d-41d5-b981-cf98ddc6a56a\" (UID: \"c8f495f1-815d-41d5-b981-cf98ddc6a56a\") "
Sep 11 03:48:42.599221 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.599194    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph" (OuterVolumeSpecName: "kube-api-access-lxwph") pod "c8f495f1-815d-41d5-b981-cf98ddc6a56a" (UID: "c8f495f1-815d-41d5-b981-cf98ddc6a56a"). InnerVolumeSpecName "kube-api-access-lxwph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:48:42.698458 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:42.698426    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxwph\" (UniqueName: \"kubernetes.io/projected/c8f495f1-815d-41d5-b981-cf98ddc6a56a-kube-api-access-lxwph\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:48:43.228260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.228175    2553 generic.go:356] "Generic (PLEG): container finished" podID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" containerID="e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28" exitCode=0
Sep 11 03:48:43.228260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.228216    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-khr67"
Sep 11 03:48:43.228507 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.228266    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-khr67" event={"ID":"c8f495f1-815d-41d5-b981-cf98ddc6a56a","Type":"ContainerDied","Data":"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"}
Sep 11 03:48:43.228507 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.228315    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-khr67" event={"ID":"c8f495f1-815d-41d5-b981-cf98ddc6a56a","Type":"ContainerDied","Data":"1a7e17d32c46fece6c50563b21d7a55baab56853c04e06a4019195da99bdbd9d"}
Sep 11 03:48:43.228507 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.228338    2553 scope.go:117] "RemoveContainer" containerID="e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"
Sep 11 03:48:43.237094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.237077    2553 scope.go:117] "RemoveContainer" containerID="e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"
Sep 11 03:48:43.237371 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:43.237350    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28\": container with ID starting with e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28 not found: ID does not exist" containerID="e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"
Sep 11 03:48:43.237426 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.237385    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28"} err="failed to get container status \"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28\": rpc error: code = NotFound desc = could not find container \"e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28\": container with ID starting with e4efde591044845daef201f2a1f7caf18d7e51356f4317e695a55a11cb85cb28 not found: ID does not exist"
Sep 11 03:48:43.244389 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.244365    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:43.247928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:43.247905    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-khr67"]
Sep 11 03:48:44.872153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:44.872121    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" path="/var/lib/kubelet/pods/c8f495f1-815d-41d5-b981-cf98ddc6a56a/volumes"
Sep 11 03:48:47.419230 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.419199    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:47.445933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.445897    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:48:47.446296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.446279    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" containerName="authorino"
Sep 11 03:48:47.446385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.446299    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" containerName="authorino"
Sep 11 03:48:47.446385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.446381    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f495f1-815d-41d5-b981-cf98ddc6a56a" containerName="authorino"
Sep 11 03:48:47.451759 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.451736    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.455004 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.454982    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 03:48:47.455103 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.455062    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 03:48:47.455349 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.455313    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w5z8x\""
Sep 11 03:48:47.456295 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.456277    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 03:48:47.462585 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.462565    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:48:47.492090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.492060    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:47.534457 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.534426    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5lmj\" (UniqueName: \"kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.534605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.534500    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.534605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.534544    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.548329 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.548297    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:47.622520 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.622485    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:47.626750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.626726    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 03:48:47.630085 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.630068    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.635733 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.635711    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4cdj\" (UniqueName: \"kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.635852 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.635760    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5lmj\" (UniqueName: \"kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.635925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.635878    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.635983 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.635956    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.636039 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.635995    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.636154 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.636043    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.636154 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:47.636086    2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 03:48:47.636259 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:47.636189    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert podName:952ec3b8-7a63-41e5-ba3a-63d278a15827 nodeName:}" failed. No retries permitted until 2026-09-11 03:48:48.136168335 +0000 UTC m=+517.778958496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert") pod "maas-controller-5f9dfc9ddc-wmm6n" (UID: "952ec3b8-7a63-41e5-ba3a-63d278a15827") : secret "maas-controller-webhook-cert" not found
Sep 11 03:48:47.638623 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.638601    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.640635 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.640612    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 03:48:47.646604 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.646579    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5lmj\" (UniqueName: \"kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:47.737154 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.737071    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.737154 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.737150    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.737343 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.737188    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4cdj\" (UniqueName: \"kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.737343 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:47.737282    2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 03:48:47.737429 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:48:47.737343    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert podName:40acae36-a37d-46b6-a6ec-80ce96ce5f8a nodeName:}" failed. No retries permitted until 2026-09-11 03:48:48.237327842 +0000 UTC m=+517.880117987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert") pod "maas-controller-66fc969cd5-n9bt7" (UID: "40acae36-a37d-46b6-a6ec-80ce96ce5f8a") : secret "maas-controller-webhook-cert" not found
Sep 11 03:48:47.739666 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.739643    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.748147 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.748122    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4cdj\" (UniqueName: \"kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:47.825252 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:47.825217    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:48.139412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.139362    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:48.141741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.141708    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") pod \"maas-controller-5f9dfc9ddc-wmm6n\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") " pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:48.240036 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.239997    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:48.242324 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.242300    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") pod \"maas-controller-66fc969cd5-n9bt7\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") " pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:48.363332 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.363259    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:48:48.541642 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.541607    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:48.635068 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.635039    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:48:48.636389 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:48:48.636367    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952ec3b8_7a63_41e5_ba3a_63d278a15827.slice/crio-f0118c19cf60244cd57e4ca15b4a3a9b4361a655fea6c544b582e128adca6a64 WatchSource:0}: Error finding container f0118c19cf60244cd57e4ca15b4a3a9b4361a655fea6c544b582e128adca6a64: Status 404 returned error can't find the container with id f0118c19cf60244cd57e4ca15b4a3a9b4361a655fea6c544b582e128adca6a64
Sep 11 03:48:48.811519 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:48.811494    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 03:48:48.814074 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:48:48.814048    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40acae36_a37d_46b6_a6ec_80ce96ce5f8a.slice/crio-1e8bed702d3b253b41cd0f6bf1af8f8592279f8ac4737842adc34da8fd74975c WatchSource:0}: Error finding container 1e8bed702d3b253b41cd0f6bf1af8f8592279f8ac4737842adc34da8fd74975c: Status 404 returned error can't find the container with id 1e8bed702d3b253b41cd0f6bf1af8f8592279f8ac4737842adc34da8fd74975c
Sep 11 03:48:49.264432 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:49.264375    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" event={"ID":"952ec3b8-7a63-41e5-ba3a-63d278a15827","Type":"ContainerStarted","Data":"f0118c19cf60244cd57e4ca15b4a3a9b4361a655fea6c544b582e128adca6a64"}
Sep 11 03:48:49.266164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:49.266128    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" event={"ID":"40acae36-a37d-46b6-a6ec-80ce96ce5f8a","Type":"ContainerStarted","Data":"1e8bed702d3b253b41cd0f6bf1af8f8592279f8ac4737842adc34da8fd74975c"}
Sep 11 03:48:51.279871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.279833    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" event={"ID":"952ec3b8-7a63-41e5-ba3a-63d278a15827","Type":"ContainerStarted","Data":"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"}
Sep 11 03:48:51.281537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.281510    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" event={"ID":"40acae36-a37d-46b6-a6ec-80ce96ce5f8a","Type":"ContainerStarted","Data":"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"}
Sep 11 03:48:51.281658 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.281637    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:48:51.326604 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.326553    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" podStartSLOduration=1.792966865 podStartE2EDuration="4.32653731s" totalImagesPullingTime="2.533570445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:48:51.300678424 +0000 UTC m=+520.943468588" watchObservedRunningTime="2026-09-11 03:48:51.32653731 +0000 UTC m=+520.969327471"
Sep 11 03:48:51.326757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.326647    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" podStartSLOduration=1.963883605 podStartE2EDuration="4.326641907s" totalImagesPullingTime="2.362758302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:48:51.325205122 +0000 UTC m=+520.967995284" watchObservedRunningTime="2026-09-11 03:48:51.326641907 +0000 UTC m=+520.969432068"
Sep 11 03:48:51.362073 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.362034    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:51.459623 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:51.459528    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:48:52.286189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:48:52.286150    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:49:02.288654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.288619    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 03:49:02.334299 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.334258    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:49:02.334524 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.334476    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" podUID="952ec3b8-7a63-41e5-ba3a-63d278a15827" containerName="manager" containerID="cri-o://6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d" gracePeriod=30
Sep 11 03:49:02.336122 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.336095    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:49:02.582108 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.582085    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:49:02.655013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.654980    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5lmj\" (UniqueName: \"kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj\") pod \"952ec3b8-7a63-41e5-ba3a-63d278a15827\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") "
Sep 11 03:49:02.655013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.655014    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs\") pod \"952ec3b8-7a63-41e5-ba3a-63d278a15827\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") "
Sep 11 03:49:02.655214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.655079    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") pod \"952ec3b8-7a63-41e5-ba3a-63d278a15827\" (UID: \"952ec3b8-7a63-41e5-ba3a-63d278a15827\") "
Sep 11 03:49:02.657246 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.657219    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj" (OuterVolumeSpecName: "kube-api-access-d5lmj") pod "952ec3b8-7a63-41e5-ba3a-63d278a15827" (UID: "952ec3b8-7a63-41e5-ba3a-63d278a15827"). InnerVolumeSpecName "kube-api-access-d5lmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:49:02.657339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.657267    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "952ec3b8-7a63-41e5-ba3a-63d278a15827" (UID: "952ec3b8-7a63-41e5-ba3a-63d278a15827"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:49:02.657339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.657289    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "952ec3b8-7a63-41e5-ba3a-63d278a15827" (UID: "952ec3b8-7a63-41e5-ba3a-63d278a15827"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:49:02.755934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.755900    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5lmj\" (UniqueName: \"kubernetes.io/projected/952ec3b8-7a63-41e5-ba3a-63d278a15827-kube-api-access-d5lmj\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:49:02.755934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.755927    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:49:02.755934 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:02.755939    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/952ec3b8-7a63-41e5-ba3a-63d278a15827-webhook-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:49:03.352178 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.352144    2553 generic.go:356] "Generic (PLEG): container finished" podID="952ec3b8-7a63-41e5-ba3a-63d278a15827" containerID="6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d" exitCode=0
Sep 11 03:49:03.352582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.352186    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n"
Sep 11 03:49:03.352582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.352221    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" event={"ID":"952ec3b8-7a63-41e5-ba3a-63d278a15827","Type":"ContainerDied","Data":"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"}
Sep 11 03:49:03.352582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.352272    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f9dfc9ddc-wmm6n" event={"ID":"952ec3b8-7a63-41e5-ba3a-63d278a15827","Type":"ContainerDied","Data":"f0118c19cf60244cd57e4ca15b4a3a9b4361a655fea6c544b582e128adca6a64"}
Sep 11 03:49:03.352582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.352294    2553 scope.go:117] "RemoveContainer" containerID="6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"
Sep 11 03:49:03.360908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.360892    2553 scope.go:117] "RemoveContainer" containerID="6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"
Sep 11 03:49:03.361209 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:49:03.361186    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d\": container with ID starting with 6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d not found: ID does not exist" containerID="6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"
Sep 11 03:49:03.361255 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.361220    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d"} err="failed to get container status \"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d\": rpc error: code = NotFound desc = could not find container \"6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d\": container with ID starting with 6cd3a30ef1257898b0dc95b885390210e9cb25ca98724f78af8a396c5502768d not found: ID does not exist"
Sep 11 03:49:03.369682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.369660    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:49:03.372926 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:03.372903    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f9dfc9ddc-wmm6n"]
Sep 11 03:49:04.871995 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:04.871963    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952ec3b8-7a63-41e5-ba3a-63d278a15827" path="/var/lib/kubelet/pods/952ec3b8-7a63-41e5-ba3a-63d278a15827/volumes"
Sep 11 03:49:16.033795 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:16.033761    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:16.085348 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:16.085312    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:25.029516 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:25.029478    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:25.128908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:25.128870    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.437423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.437386    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.540427 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.540392    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.562188 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.562160    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 03:49:26.562518 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.562503    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952ec3b8-7a63-41e5-ba3a-63d278a15827" containerName="manager"
Sep 11 03:49:26.562571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.562521    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="952ec3b8-7a63-41e5-ba3a-63d278a15827" containerName="manager"
Sep 11 03:49:26.562610 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.562588    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="952ec3b8-7a63-41e5-ba3a-63d278a15827" containerName="manager"
Sep 11 03:49:26.568573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.568554    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.573259 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.573233    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 03:49:26.573437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.573233    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fcpfs\""
Sep 11 03:49:26.573437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.573253    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 03:49:26.577227 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.577207    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 03:49:26.640664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.640629    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.668721 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.668689    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5dhn\" (UniqueName: \"kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.668889 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.668728    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.668889 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.668748    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.668889 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.668865    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.770125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.770086    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.770310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.770260    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5dhn\" (UniqueName: \"kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.770385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.770327    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.770385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.770350    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.770879 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.770856    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.772912 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.772889    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.772912 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.772906    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.778041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.778023    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5dhn\" (UniqueName: \"kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn\") pod \"maas-api-577f88484f-svdhb\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") " pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.879757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.879724    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:26.900094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.900056    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.988857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.988805    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:26.999296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:26.999255    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"]
Sep 11 03:49:27.004894 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.004859    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.007551 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.007533    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 03:49:27.011710 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.011688    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"]
Sep 11 03:49:27.158485 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.158454    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 03:49:27.159370 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:49:27.159343    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c9ae90_9e6a_4ebe_8c98_1e925e1b61fd.slice/crio-b57251823c2477706727018c9976275e24e4a61c21a6f3c64c7be533484c26cf WatchSource:0}: Error finding container b57251823c2477706727018c9976275e24e4a61c21a6f3c64c7be533484c26cf: Status 404 returned error can't find the container with id b57251823c2477706727018c9976275e24e4a61c21a6f3c64c7be533484c26cf
Sep 11 03:49:27.174077 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.174051    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ef199-4502-4e5c-9005-7aa9927b9b09-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.174193 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.174174    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwqwh\" (UniqueName: \"kubernetes.io/projected/b66ef199-4502-4e5c-9005-7aa9927b9b09-kube-api-access-xwqwh\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.275289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.275254    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwqwh\" (UniqueName: \"kubernetes.io/projected/b66ef199-4502-4e5c-9005-7aa9927b9b09-kube-api-access-xwqwh\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.275470 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.275325    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ef199-4502-4e5c-9005-7aa9927b9b09-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.276262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.276238    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66ef199-4502-4e5c-9005-7aa9927b9b09-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.288351 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.288322    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwqwh\" (UniqueName: \"kubernetes.io/projected/b66ef199-4502-4e5c-9005-7aa9927b9b09-kube-api-access-xwqwh\") pod \"payload-pre-processing-59fbc47bf8-k796r\" (UID: \"b66ef199-4502-4e5c-9005-7aa9927b9b09\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.317141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.317115    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:27.475404 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.475319    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" event={"ID":"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd","Type":"ContainerStarted","Data":"b57251823c2477706727018c9976275e24e4a61c21a6f3c64c7be533484c26cf"}
Sep 11 03:49:27.587016 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:27.586990    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"]
Sep 11 03:49:27.588254 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:49:27.588227    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66ef199_4502_4e5c_9005_7aa9927b9b09.slice/crio-a25d4f2361f4391adf4583ceeb1f3c16f4c53785fd5a4d6945edb7405e873e56 WatchSource:0}: Error finding container a25d4f2361f4391adf4583ceeb1f3c16f4c53785fd5a4d6945edb7405e873e56: Status 404 returned error can't find the container with id a25d4f2361f4391adf4583ceeb1f3c16f4c53785fd5a4d6945edb7405e873e56
Sep 11 03:49:28.485605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:28.485525    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r" event={"ID":"b66ef199-4502-4e5c-9005-7aa9927b9b09","Type":"ContainerStarted","Data":"a25d4f2361f4391adf4583ceeb1f3c16f4c53785fd5a4d6945edb7405e873e56"}
Sep 11 03:49:30.497602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.497569    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" event={"ID":"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd","Type":"ContainerStarted","Data":"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"}
Sep 11 03:49:30.498100 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.497626    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:30.499141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.499115    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r" event={"ID":"b66ef199-4502-4e5c-9005-7aa9927b9b09","Type":"ContainerStarted","Data":"47a71f4d70e271c07a5e098f8844caa6080aae9e80581ba0a268d6b7c2e087f6"}
Sep 11 03:49:30.499240 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.499164    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:30.516287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.516244    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" podStartSLOduration=1.9653424579999998 podStartE2EDuration="4.516231404s" totalImagesPullingTime="2.550888946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:49:30.51497252 +0000 UTC m=+560.157762682" watchObservedRunningTime="2026-09-11 03:49:30.516231404 +0000 UTC m=+560.159021566"
Sep 11 03:49:30.532844 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:30.532775    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r" podStartSLOduration=2.404892413 podStartE2EDuration="4.532762245s" totalImagesPullingTime="2.127869832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:49:30.5303436 +0000 UTC m=+560.173133759" watchObservedRunningTime="2026-09-11 03:49:30.532762245 +0000 UTC m=+560.175552407"
Sep 11 03:49:36.512908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.512879    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 03:49:36.534127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.534059    2553 generic.go:356] "Generic (PLEG): container finished" podID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerID="39f692d5b247e8132a293682493faa0c882ac2f57f3f8a4844324867bc73eb94" exitCode=0
Sep 11 03:49:36.534127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.534101    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" event={"ID":"6d9335c5-6a0b-4020-98de-d4eaafdfba1b","Type":"ContainerDied","Data":"39f692d5b247e8132a293682493faa0c882ac2f57f3f8a4844324867bc73eb94"}
Sep 11 03:49:36.534471 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.534454    2553 scope.go:117] "RemoveContainer" containerID="39f692d5b247e8132a293682493faa0c882ac2f57f3f8a4844324867bc73eb94"
Sep 11 03:49:36.535454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.535423    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:49:36.535546 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.535491    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:49:36.535546 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:36.535520    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:49:37.543364 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.543325    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" event={"ID":"6d9335c5-6a0b-4020-98de-d4eaafdfba1b","Type":"ContainerStarted","Data":"8b57527c3af1f837985f0de1e70f752e916d0fed1cf73659cccbd3ee705b9865"}
Sep 11 03:49:37.545493 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.545460    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:37.624150 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.624117    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:49:37.957678 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.957581    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:49:37.957957 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.957921    2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-5zqtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 03:49:37.958045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:37.957990    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podUID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 03:49:38.957860 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:38.957803    2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-5zqtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 03:49:38.959093 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:38.959060    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podUID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 03:49:39.957988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:39.957952    2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-5zqtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 03:49:39.958568 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:39.958533    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podUID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 03:49:40.959291 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:40.959252    2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-5zqtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 03:49:40.959942 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:40.959906    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podUID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 03:49:41.509613 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:41.509586    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k796r"
Sep 11 03:49:41.958051 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:41.958013    2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-5zqtx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 03:49:41.958356 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:41.958326    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx" podUID="6d9335c5-6a0b-4020-98de-d4eaafdfba1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 03:49:42.962125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:42.962093    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:49:42.962510 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:42.962380    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:49:42.963172 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:42.963154    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-5zqtx"
Sep 11 03:49:54.690432 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:49:54.690391    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:07.601537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.601499    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"]
Sep 11 03:50:07.605116 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.605095    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.609510 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.609486    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d5247\""
Sep 11 03:50:07.609633 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.609513    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 03:50:07.609633 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.609557    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 03:50:07.609633 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.609558    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 03:50:07.613877 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.613853    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"]
Sep 11 03:50:07.631010 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.630976    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:07.708589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708560    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.708589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708590    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh2pr\" (UniqueName: \"kubernetes.io/projected/38144661-3419-4dd5-a3c8-2827c7d608f6-kube-api-access-vh2pr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.708805 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708612    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.708805 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708690    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.708805 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708791    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.708974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.708825    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38144661-3419-4dd5-a3c8-2827c7d608f6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809436    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809477    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38144661-3419-4dd5-a3c8-2827c7d608f6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809511    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809527    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh2pr\" (UniqueName: \"kubernetes.io/projected/38144661-3419-4dd5-a3c8-2827c7d608f6-kube-api-access-vh2pr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809545    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.809741 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809569    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.810006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809965    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.810006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.809983    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.810090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.810017    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.811950 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.811922    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38144661-3419-4dd5-a3c8-2827c7d608f6-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.812129 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.812100    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38144661-3419-4dd5-a3c8-2827c7d608f6-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.819740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.819719    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh2pr\" (UniqueName: \"kubernetes.io/projected/38144661-3419-4dd5-a3c8-2827c7d608f6-kube-api-access-vh2pr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d2xxq\" (UID: \"38144661-3419-4dd5-a3c8-2827c7d608f6\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:07.917251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:07.917148    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:08.208853 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:08.205935    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"]
Sep 11 03:50:08.716924 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:08.716877    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq" event={"ID":"38144661-3419-4dd5-a3c8-2827c7d608f6","Type":"ContainerStarted","Data":"a2b653433a6ece40fa1b1fcb0fd27656e51eb66bb0fea61340175ad333553677"}
Sep 11 03:50:09.723702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:09.723607    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq" event={"ID":"38144661-3419-4dd5-a3c8-2827c7d608f6","Type":"ContainerStarted","Data":"6b92da467ca3942733d4195a0e15f1c15c428c1597825d527d1cf19dc285f819"}
Sep 11 03:50:09.745782 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:09.745733    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq" podStartSLOduration=1.626184602 podStartE2EDuration="2.745718281s" totalImagesPullingTime="1.119533679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:50:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:50:09.743677747 +0000 UTC m=+599.386467921" watchObservedRunningTime="2026-09-11 03:50:09.745718281 +0000 UTC m=+599.388508484"
Sep 11 03:50:10.818714 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:10.818687    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:50:10.820075 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:10.820053    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:50:12.999961 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:12.999924    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:17.918205 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:17.918172    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:17.918582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:17.918502    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:17.930928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:17.930899    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:18.786405 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:18.786377    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d2xxq"
Sep 11 03:50:20.892631 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:20.892597    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"]
Sep 11 03:50:20.896581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:20.896560    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:20.899252 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:20.899223    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 11 03:50:20.907736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:20.907704    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"]
Sep 11 03:50:20.922130 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:20.922063    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:21.025702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025665    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.025702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025704    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.025989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025795    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.025989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025867    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.025989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025908    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64zq\" (UniqueName: \"kubernetes.io/projected/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-kube-api-access-k64zq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.025989 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.025936    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126424    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126457    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126703 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126507    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126703 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126526    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126703 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126550    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64zq\" (UniqueName: \"kubernetes.io/projected/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-kube-api-access-k64zq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126703 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126571    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.126953 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.126916    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.127038 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.127014    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.127038 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.127031    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.128748 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.128718    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.129075 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.129054    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.138268 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.138239    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64zq\" (UniqueName: \"kubernetes.io/projected/a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9-kube-api-access-k64zq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dl9kz\" (UID: \"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.210465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.210383    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:21.481779 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.481747    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"]
Sep 11 03:50:21.482804 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:50:21.482776    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00dc0cd_08e0_4ba9_a250_4d9b4483a6b9.slice/crio-19b4d014669dba73cb0632035b7f786dcb526c8be437e410b842578752fb4587 WatchSource:0}: Error finding container 19b4d014669dba73cb0632035b7f786dcb526c8be437e410b842578752fb4587: Status 404 returned error can't find the container with id 19b4d014669dba73cb0632035b7f786dcb526c8be437e410b842578752fb4587
Sep 11 03:50:21.484274 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.484257    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 03:50:21.792411 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.792375    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz" event={"ID":"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9","Type":"ContainerStarted","Data":"4d49d9deb463e6113e463884a00905001da2fe5e0eac71996bfc62f1b072b67c"}
Sep 11 03:50:21.792564 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.792428    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz" event={"ID":"a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9","Type":"ContainerStarted","Data":"19b4d014669dba73cb0632035b7f786dcb526c8be437e410b842578752fb4587"}
Sep 11 03:50:21.810762 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:21.810714    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz" podStartSLOduration=1.618857504 podStartE2EDuration="1.810699306s" totalImagesPullingTime="191.841802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:50:21.808539997 +0000 UTC m=+611.451330159" watchObservedRunningTime="2026-09-11 03:50:21.810699306 +0000 UTC m=+611.453489467"
Sep 11 03:50:26.588884 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.588797    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"]
Sep 11 03:50:26.592964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.592942    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.595861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.595842    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 03:50:26.605145 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.605121    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"]
Sep 11 03:50:26.624723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.624699    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:26.676604 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676577    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82cpc\" (UniqueName: \"kubernetes.io/projected/eb81cbb1-d7da-41f5-964b-d12230e72f74-kube-api-access-82cpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.676736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676627    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.676736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676661    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.676736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676684    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.676736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676704    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.676736 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.676721    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb81cbb1-d7da-41f5-964b-d12230e72f74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778118 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778084    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778279 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778124    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778321 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778286    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778359 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778318    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb81cbb1-d7da-41f5-964b-d12230e72f74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778383    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82cpc\" (UniqueName: \"kubernetes.io/projected/eb81cbb1-d7da-41f5-964b-d12230e72f74-kube-api-access-82cpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778467 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778449    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778516    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778611 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778458    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.778805 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.778786    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.780624 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.780600    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb81cbb1-d7da-41f5-964b-d12230e72f74-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.780849 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.780829    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb81cbb1-d7da-41f5-964b-d12230e72f74-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.786416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.786391    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82cpc\" (UniqueName: \"kubernetes.io/projected/eb81cbb1-d7da-41f5-964b-d12230e72f74-kube-api-access-82cpc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq\" (UID: \"eb81cbb1-d7da-41f5-964b-d12230e72f74\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:26.904465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:26.904389    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:27.192262 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:27.192234    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"]
Sep 11 03:50:27.193396 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:50:27.193368    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb81cbb1_d7da_41f5_964b_d12230e72f74.slice/crio-596d6e657615e4edb2a1cb564d05e842d92a240288e654986076b3de4aa18658 WatchSource:0}: Error finding container 596d6e657615e4edb2a1cb564d05e842d92a240288e654986076b3de4aa18658: Status 404 returned error can't find the container with id 596d6e657615e4edb2a1cb564d05e842d92a240288e654986076b3de4aa18658
Sep 11 03:50:27.825714 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:27.825680    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq" event={"ID":"eb81cbb1-d7da-41f5-964b-d12230e72f74","Type":"ContainerStarted","Data":"6d953e2e9fb73143d38dbfb217bd9cf7ffb530709fd0b874a672bb9eb8885e8e"}
Sep 11 03:50:27.826128 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:27.825730    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq" event={"ID":"eb81cbb1-d7da-41f5-964b-d12230e72f74","Type":"ContainerStarted","Data":"596d6e657615e4edb2a1cb564d05e842d92a240288e654986076b3de4aa18658"}
Sep 11 03:50:27.847196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:27.847145    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq" podStartSLOduration=1.680866341 podStartE2EDuration="1.847128035s" totalImagesPullingTime="166.261694ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:50:27.845978157 +0000 UTC m=+617.488768320" watchObservedRunningTime="2026-09-11 03:50:27.847128035 +0000 UTC m=+617.489918246"
Sep 11 03:50:31.211239 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:31.211192    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:31.211618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:31.211266    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:31.223740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:31.223713    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:31.859028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:31.858997    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dl9kz"
Sep 11 03:50:36.888411 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:36.888318    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:50:36.905248 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:36.905217    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:36.905374 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:36.905270    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:36.917498 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:36.917476    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:37.893851 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:37.893803    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq"
Sep 11 03:50:50.016739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:50:50.016699    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:51:05.369287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:51:05.369249    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:51:05.473031 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:51:05.472992    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:51:23.785535 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:51:23.785502    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:30.858743 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:30.858702    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:30.959141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:30.959104    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:32.191080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:32.191035    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:32.288901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:32.288864    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:36.419859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:36.419801    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:36.520373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:36.520338    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:48.744689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:48.744650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:48.854756 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:48.854716    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:49.050788 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.050751    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:52:49.054589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.054561    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.057139 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.057113    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4c62f1-openshift-default-dockercfg-kfdht\""
Sep 11 03:52:49.074527 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.073910    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:52:49.074705 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.074674    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:49.243222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243186    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243222 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243226    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243243    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243270    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243505 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243431    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243505 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243470    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243505 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243497    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243534    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.243602 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.243559    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k85jl\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.251577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.251542    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:49.345015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.344936    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.344975    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.344994    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345064    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345100    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k85jl\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345156    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345182    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345208    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345301 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345253    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345383    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345422    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345642    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345717    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.345892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.345807    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.347560 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.347540    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.347702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.347683    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.357971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.357945    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k85jl\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.358090 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.357974    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token\") pod \"e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") " pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.373950 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.373926    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:49.651021 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.650997    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:52:49.652538 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:52:49.652511    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2693b79f_4964_42f7_a46c_553d7ef3a3c1.slice/crio-820ce6b8e5e5552e399b308c5b0efe9ffd8da2ccf449f26ac482bcf5adcff9e1 WatchSource:0}: Error finding container 820ce6b8e5e5552e399b308c5b0efe9ffd8da2ccf449f26ac482bcf5adcff9e1: Status 404 returned error can't find the container with id 820ce6b8e5e5552e399b308c5b0efe9ffd8da2ccf449f26ac482bcf5adcff9e1
Sep 11 03:52:49.654695 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.654662    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:49.654778 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.654726    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:49.654778 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:49.654755    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:50.259270 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.259220    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:50.357871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.357837    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:50.586203 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.586169    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerStarted","Data":"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"}
Sep 11 03:52:50.586375 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.586219    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerStarted","Data":"820ce6b8e5e5552e399b308c5b0efe9ffd8da2ccf449f26ac482bcf5adcff9e1"}
Sep 11 03:52:50.607666 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.607612    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podStartSLOduration=1.6075979980000001 podStartE2EDuration="1.607597998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:52:50.604998667 +0000 UTC m=+760.247788840" watchObservedRunningTime="2026-09-11 03:52:50.607597998 +0000 UTC m=+760.250388159"
Sep 11 03:52:50.700331 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:50.700295    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:51.374347 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:51.374316    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:51.378964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:51.378939    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:51.592206 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:51.592174    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:51.593072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:51.593054    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:52:52.616569 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:52.616528    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:52.715233 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:52.715203    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:53.905589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.905554    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:52:53.909455 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.909433    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:53.911929 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.911909    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4c62f1-serving-cert\""
Sep 11 03:52:53.918320 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.918294    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:52:53.940414 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.940382    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:53.986308 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.986273    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jckn\" (UniqueName: \"kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:53.986438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.986325    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:53.986438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.986370    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:53.986438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:53.986405    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.040860 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.040806    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:54.086905 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.086867    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.087080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.086932    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jckn\" (UniqueName: \"kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.087080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.086980    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.087080 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:52:54.087010    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-serving-cert: secret "maas-api-e2e-inf-a-4c62f1-serving-cert" not found
Sep 11 03:52:54.087080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.087006    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.087283 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:52:54.087087    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls podName:639894f8-1b35-4dab-853a-5d80bde40795 nodeName:}" failed. No retries permitted until 2026-09-11 03:52:54.58706559 +0000 UTC m=+764.229855747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls") pod "maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" (UID: "639894f8-1b35-4dab-853a-5d80bde40795") : secret "maas-api-e2e-inf-a-4c62f1-serving-cert" not found
Sep 11 03:52:54.087632 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.087612    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.089405 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.089384    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.095276 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.095250    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jckn\" (UniqueName: \"kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.295967 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.295928    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:54.398004 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.397956    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:54.483935 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.483899    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:54.591583 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.591505    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.594015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.593990    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") pod \"maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:54.823144 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:54.823110    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:55.102162 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:55.102133    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:52:55.103888 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:52:55.103859    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639894f8_1b35_4dab_853a_5d80bde40795.slice/crio-507c9fcf4fa94d843c6aeb3df552f05ca22659ffd8a81179bf398b1d730d1f89 WatchSource:0}: Error finding container 507c9fcf4fa94d843c6aeb3df552f05ca22659ffd8a81179bf398b1d730d1f89: Status 404 returned error can't find the container with id 507c9fcf4fa94d843c6aeb3df552f05ca22659ffd8a81179bf398b1d730d1f89
Sep 11 03:52:55.618473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:55.618447    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" event={"ID":"639894f8-1b35-4dab-853a-5d80bde40795","Type":"ContainerStarted","Data":"507c9fcf4fa94d843c6aeb3df552f05ca22659ffd8a81179bf398b1d730d1f89"}
Sep 11 03:52:55.788077 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:55.788039    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:56.626096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.626060    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" event={"ID":"639894f8-1b35-4dab-853a-5d80bde40795","Type":"ContainerStarted","Data":"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"}
Sep 11 03:52:56.626481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.626186    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:52:56.644659 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.644614    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" podStartSLOduration=3.175480408 podStartE2EDuration="3.644600854s" totalImagesPullingTime="469.120446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:52:56.642077135 +0000 UTC m=+766.284867296" watchObservedRunningTime="2026-09-11 03:52:56.644600854 +0000 UTC m=+766.287391016"
Sep 11 03:52:56.683937 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.683904    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:56.821190 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.821151    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:56.922629 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:56.922546    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.063384 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.063348    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.350992 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.350960    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:52:57.354880 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.354858    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.357699 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.357674    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1eeb81b2-openshift-default-dockercfg-97lnz\""
Sep 11 03:52:57.366478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.366453    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:52:57.375722 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.375694    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.412205 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412180    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412226    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412242    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412263    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412332    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412368    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412415 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412388    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dp68\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412519 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412442    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.412519 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.412487    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513478    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513518    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dp68\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513540    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513676    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513757 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513721    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513967 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513786    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513967 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513912    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.513967 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513953    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514131 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.513967    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514131 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.514057    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514218 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.514125    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514218 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.514158    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514218 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.514193    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.514312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.514292    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.516399 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.516373    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.516701 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.516682    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.521233 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.521207    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.522295 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.522275    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dp68\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.522437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.522416    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token\") pod \"e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") " pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.580877 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.580839    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.666593 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.666510    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:57.868520 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.868466    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:57.941305 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.941272    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:52:57.942577 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:52:57.942552    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa1a9cc_437d_4498_be61_917bdd15cdff.slice/crio-7a0954297e811b176b9aafedad83d055f88011519da7aa13331bb5541790b094 WatchSource:0}: Error finding container 7a0954297e811b176b9aafedad83d055f88011519da7aa13331bb5541790b094: Status 404 returned error can't find the container with id 7a0954297e811b176b9aafedad83d055f88011519da7aa13331bb5541790b094
Sep 11 03:52:57.945168 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.945138    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:57.945241 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.945199    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:57.945241 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:57.945228    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:52:58.433680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.433650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:52:58.638997 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.638960    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerStarted","Data":"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"}
Sep 11 03:52:58.639175 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.639017    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerStarted","Data":"7a0954297e811b176b9aafedad83d055f88011519da7aa13331bb5541790b094"}
Sep 11 03:52:58.661320 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.661270    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podStartSLOduration=1.661255041 podStartE2EDuration="1.661255041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:52:58.659022325 +0000 UTC m=+768.301812500" watchObservedRunningTime="2026-09-11 03:52:58.661255041 +0000 UTC m=+768.304045204"
Sep 11 03:52:58.667143 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.667118    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:58.672121 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:58.672098    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:59.649694 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:59.649659    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:52:59.654279 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:52:59.654247    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:00.042873 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:00.042832    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:00.168498 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:00.168447    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:01.257838 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:01.256854    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:01.345737 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:01.345705    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:01.484338 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:01.484300    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:01.715413 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:01.715376    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:01.817310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:01.817279    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:02.645046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:02.644718    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:53:03.642835 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:03.642789    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:03.747289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:03.747248    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:03.846928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:03.846892    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:04.646668 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:04.646630    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:05.605748 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:05.605692    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:06.703750 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.703712    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:53:06.708349 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.707900    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.710503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.710477    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-fd0def-serving-cert\""
Sep 11 03:53:06.719089 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.719066    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:53:06.767031 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.766790    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:06.828925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.828886    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.829164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.828950    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxq9\" (UniqueName: \"kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.829164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.829023    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.829164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.829080    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.834142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.834110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:06.930289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.930257    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxq9\" (UniqueName: \"kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.930478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.930317    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.930551 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.930524    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.930687 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:06.930668    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-serving-cert: secret "maas-api-e2e-autores-fd0def-serving-cert" not found
Sep 11 03:53:06.930840 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:06.930796    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls podName:1c033029-c403-4755-a756-f433ad9fa80c nodeName:}" failed. No retries permitted until 2026-09-11 03:53:07.430769941 +0000 UTC m=+777.073560096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls") pod "maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" (UID: "1c033029-c403-4755-a756-f433ad9fa80c") : secret "maas-api-e2e-autores-fd0def-serving-cert" not found
Sep 11 03:53:06.930908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.930891    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.930990 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.930972    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.933396 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.933374    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:06.947149 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:06.947111    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxq9\" (UniqueName: \"kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:07.061967 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.061889    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.256512 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.256410    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.353622 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.353526    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.435238 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.435200    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:07.445836 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.445520    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") pod \"maas-api-e2e-autores-fd0def-64c5bd75c8-26q24\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:07.458280 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.458242    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.593447 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.593410    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 03:53:07.593723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.593472    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 03:53:07.597056 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.597023    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.622180 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.622110    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:07.718464 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.718412    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.811465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.811426    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:07.931171 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.931144    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:53:07.933709 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:53:07.933663    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c033029_c403_4755_a756_f433ad9fa80c.slice/crio-8c38820a087d7fb1165bf797c571b129e59655d35fd5cf3f668457d83c838522 WatchSource:0}: Error finding container 8c38820a087d7fb1165bf797c571b129e59655d35fd5cf3f668457d83c838522: Status 404 returned error can't find the container with id 8c38820a087d7fb1165bf797c571b129e59655d35fd5cf3f668457d83c838522
Sep 11 03:53:07.989789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:07.989760    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:08.092131 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.092100    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:08.257652 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.257573    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:08.717830 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.717789    2553 generic.go:356] "Generic (PLEG): container finished" podID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerID="96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a" exitCode=0
Sep 11 03:53:08.718006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.717862    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerDied","Data":"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"}
Sep 11 03:53:08.718324 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.718307    2553 scope.go:117] "RemoveContainer" containerID="96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"
Sep 11 03:53:08.719455 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719409    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:08.719809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719513    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:08.719809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719529    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" event={"ID":"1c033029-c403-4755-a756-f433ad9fa80c","Type":"ContainerStarted","Data":"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"}
Sep 11 03:53:08.719809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719572    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" event={"ID":"1c033029-c403-4755-a756-f433ad9fa80c","Type":"ContainerStarted","Data":"8c38820a087d7fb1165bf797c571b129e59655d35fd5cf3f668457d83c838522"}
Sep 11 03:53:08.719809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719574    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:08.719809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.719672    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:08.781663 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.781098    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" podStartSLOduration=2.4819581299999998 podStartE2EDuration="2.781072688s" totalImagesPullingTime="299.114558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:53:08.780551633 +0000 UTC m=+778.423341795" watchObservedRunningTime="2026-09-11 03:53:08.781072688 +0000 UTC m=+778.423862853"
Sep 11 03:53:08.953264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:08.953126    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:09.642789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:09.642747    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:09.729431 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:09.729396    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerStarted","Data":"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"}
Sep 11 03:53:10.374579 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.374538    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:53:10.374793 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.374681    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:10.374793 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.374741    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:10.428402 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.428350    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:10.479577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.479535    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:10.569694 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:10.569636    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:11.374954 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:11.374906    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:11.375465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:11.374993    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:12.162070 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:12.161906    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:12.375200 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:12.375155    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:12.375557 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:12.375214    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:13.374552 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:13.374507    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:13.374864 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:13.374799    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:13.569545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:13.569503    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:14.007459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.007360    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:14.107138 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.107097    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:14.375072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.375036    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:14.375248 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.375102    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:14.627092 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.627004    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:14.735530 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:14.735502    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:53:15.375067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:15.375021    2553 patch_prober.go:28] interesting pod/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 03:53:15.375269 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:15.375099    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 03:53:15.691992 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:15.691954    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 03:53:15.693464 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:15.692043    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 03:53:15.713539 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:15.713504    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:16.378600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.378568    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:53:16.378880 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.378846    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:53:16.379465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.379446    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:53:16.508966 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.508932    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:16.789004 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.788973    2553 generic.go:356] "Generic (PLEG): container finished" podID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerID="9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01" exitCode=0
Sep 11 03:53:16.789412 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.789047    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerDied","Data":"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"}
Sep 11 03:53:16.789643 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.789622    2553 scope.go:117] "RemoveContainer" containerID="9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"
Sep 11 03:53:16.790872 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.790847    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:16.790958 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.790918    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:16.790958 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:16.790946    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:53:17.128506 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.128409    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:53:17.131072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.131046    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.134105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.134086    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8efac4-serving-cert\""
Sep 11 03:53:17.144714 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.144693    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:53:17.165838 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.165785    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:17.262652 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.262619    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:17.276890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.276844    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.276890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.276886    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbd6\" (UniqueName: \"kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.277073 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.276913    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.277073 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.277040    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.378434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.378406    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.378434 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.378438    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbd6\" (UniqueName: \"kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.378711 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.378458    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.378711 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.378488    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.378711 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:17.378657    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-serving-cert: secret "maas-api-e2e-inf-b-8efac4-serving-cert" not found
Sep 11 03:53:17.378929 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:17.378730    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls podName:6c80d8ef-0aaa-4d6a-acc4-8012ad384218 nodeName:}" failed. No retries permitted until 2026-09-11 03:53:17.87871076 +0000 UTC m=+787.521500904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls") pod "maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" (UID: "6c80d8ef-0aaa-4d6a-acc4-8012ad384218") : secret "maas-api-e2e-inf-b-8efac4-serving-cert" not found
Sep 11 03:53:17.379121 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.379102    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.380947 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.380923    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.386710 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.386690    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbd6\" (UniqueName: \"kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.615420 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.615360    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:17.713163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.713071    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:17.796594 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.796561    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerStarted","Data":"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"}
Sep 11 03:53:17.885504 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.885468    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.888199 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.888170    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") pod \"maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:17.931142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:17.931109    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:18.043754 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.043716    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:18.153784 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.153656    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:18.334753 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.334724    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:53:18.335494 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:53:18.335472    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c80d8ef_0aaa_4d6a_acc4_8012ad384218.slice/crio-ff2656be6859926e6028d386f10107997c45b20abf3e3817eec2890b1031489d WatchSource:0}: Error finding container ff2656be6859926e6028d386f10107997c45b20abf3e3817eec2890b1031489d: Status 404 returned error can't find the container with id ff2656be6859926e6028d386f10107997c45b20abf3e3817eec2890b1031489d
Sep 11 03:53:18.488898 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.488866    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:18.667192 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.667115    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:18.667338 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.667309    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 03:53:18.667388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.667354    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 03:53:18.808209 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.808172    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" event={"ID":"6c80d8ef-0aaa-4d6a-acc4-8012ad384218","Type":"ContainerStarted","Data":"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"}
Sep 11 03:53:18.808680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.808242    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" event={"ID":"6c80d8ef-0aaa-4d6a-acc4-8012ad384218","Type":"ContainerStarted","Data":"ff2656be6859926e6028d386f10107997c45b20abf3e3817eec2890b1031489d"}
Sep 11 03:53:18.808680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.808421    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:18.832287 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.832239    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" podStartSLOduration=1.4722330590000001 podStartE2EDuration="1.832223673s" totalImagesPullingTime="359.990614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:53:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:53:18.829764247 +0000 UTC m=+788.472554421" watchObservedRunningTime="2026-09-11 03:53:18.832223673 +0000 UTC m=+788.475013835"
Sep 11 03:53:18.936027 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:18.935939    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:19.581346 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:19.581304    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:19.667839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:19.667407    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 03:53:19.667839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:19.667491    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 03:53:20.345907 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:20.345826    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:20.667348 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:20.667255    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 03:53:20.667348 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:20.667323    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 03:53:21.666986 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:21.666944    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 03:53:21.667399 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:21.667001    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 03:53:22.118740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.118699    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:22.632134 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.632093    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:22.667577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.667534    2553 patch_prober.go:28] interesting pod/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 03:53:22.668041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.667587    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 03:53:22.732923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.732884    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:22.855776 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:22.855720    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:23.670860 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:23.670799    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:23.671278 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:23.671260    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:23.672006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:23.671974    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:24.821562 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:24.821526    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:53:25.912378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:25.912344    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:25.993284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:25.993252    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.050545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.050510    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.150437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.150380    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.223067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.222979    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:53:26.223529 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.223492    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy" containerID="cri-o://719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733" gracePeriod=30
Sep 11 03:53:26.296341 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.296299    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.491756 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.491667    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.736063 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.736027    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.834903 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.834867    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:26.961605 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:26.961568    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:27.268131 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:27.268101    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:27.423091 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:27.423056    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:27.867977 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:27.867944    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:28.206171 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:28.206090    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:28.789867 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:28.789832    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.025573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.025538    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.289332 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.289294    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.397948 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.397910    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.489590 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.489555    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.664324 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.664238    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.768288 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.768252    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.864919 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.864875    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:29.967128 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:29.967035    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:30.831879 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:30.831848    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:31.547428 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.547395    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:31.582283 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.582230    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:31.628932 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.628896    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.628944    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629012    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629094 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629081    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629146    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dp68\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629205    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629179    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:53:31.629392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629248    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:53:31.629392 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629370    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629499 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629425    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629499 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629446    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token\") pod \"1fa1a9cc-437d-4498-be61-917bdd15cdff\" (UID: \"1fa1a9cc-437d-4498-be61-917bdd15cdff\") "
Sep 11 03:53:31.629499 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629461    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:53:31.629647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629539    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:53:31.629716 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629690    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:53:31.629801 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629785    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.629886 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629803    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.629886 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629839    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fa1a9cc-437d-4498-be61-917bdd15cdff-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.629886 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629853    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.629886 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.629862    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.631731 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.631701    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:53:31.631869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.631847    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 03:53:31.631869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.631855    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68" (OuterVolumeSpecName: "kube-api-access-2dp68") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "kube-api-access-2dp68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:53:31.632038 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.632022    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fa1a9cc-437d-4498-be61-917bdd15cdff" (UID: "1fa1a9cc-437d-4498-be61-917bdd15cdff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:53:31.649490 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.649464    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:31.730618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.730583    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dp68\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-kube-api-access-2dp68\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.730618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.730615    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.730859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.730628    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.730859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.730640    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fa1a9cc-437d-4498-be61-917bdd15cdff-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:53:31.750901 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.750857    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:31.893444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.893371    2553 generic.go:356] "Generic (PLEG): container finished" podID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerID="719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733" exitCode=0
Sep 11 03:53:31.893444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.893426    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"
Sep 11 03:53:31.893904 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.893455    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerDied","Data":"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"}
Sep 11 03:53:31.893904 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.893509    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq" event={"ID":"1fa1a9cc-437d-4498-be61-917bdd15cdff","Type":"ContainerDied","Data":"7a0954297e811b176b9aafedad83d055f88011519da7aa13331bb5541790b094"}
Sep 11 03:53:31.893904 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.893538    2553 scope.go:117] "RemoveContainer" containerID="719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"
Sep 11 03:53:31.903923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.903904    2553 scope.go:117] "RemoveContainer" containerID="9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"
Sep 11 03:53:31.912538 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.912521    2553 scope.go:117] "RemoveContainer" containerID="719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"
Sep 11 03:53:31.912889 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:31.912869    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733\": container with ID starting with 719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733 not found: ID does not exist" containerID="719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"
Sep 11 03:53:31.912988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.912893    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733"} err="failed to get container status \"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733\": rpc error: code = NotFound desc = could not find container \"719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733\": container with ID starting with 719fa5a42a05a429dc34ad04b9b3ac32e81301bb9181d511d9d963e055d18733 not found: ID does not exist"
Sep 11 03:53:31.912988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.912908    2553 scope.go:117] "RemoveContainer" containerID="9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"
Sep 11 03:53:31.913155 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:53:31.913137    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01\": container with ID starting with 9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01 not found: ID does not exist" containerID="9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"
Sep 11 03:53:31.913216 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.913160    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01"} err="failed to get container status \"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01\": rpc error: code = NotFound desc = could not find container \"9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01\": container with ID starting with 9145fb7945f2fa30a614850edfa22570b681b3c8da8cd0eb3bd10159c7a0ab01 not found: ID does not exist"
Sep 11 03:53:31.918626 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.918601    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:53:31.920962 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.920938    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1eeb81b2-openshift-default-d68fd5764-7hwtq"]
Sep 11 03:53:31.986534 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:31.986499    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:32.091740 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:32.091705    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:32.349025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:32.348982    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:32.873498 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:32.873464    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" path="/var/lib/kubelet/pods/1fa1a9cc-437d-4498-be61-917bdd15cdff/volumes"
Sep 11 03:53:33.010850 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:33.010799    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:35.217119 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:35.217089    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:35.316640 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:35.316606    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:37.517609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:37.517573    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:38.123667 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:38.123626    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:52.355421 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:52.355361    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:53.712248 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:53.712214    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:54.424931 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:54.424899    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:55.214346 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:55.214313    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:55.925435 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:55.925402    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:56.534642 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:56.534595    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:57.308739 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:57.308692    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:53:57.916652 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:53:57.916621    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:00.599902 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:00.599856    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:01.008783 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:01.008684    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:01.642433 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:01.642402    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:01.742969 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:01.742934    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:01.853649 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:01.853616    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:01.939401 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:01.939314    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:02.020642 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:02.020603    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:02.332536 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:02.332474    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:02.494010 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:02.493957    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:02.779248 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:02.779100    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:02.933510 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:02.933469    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:03.034127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:03.034022    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:03.725237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:03.725202    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:04.330811 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:04.330775    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:05.102339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:05.102306    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:05.656744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:05.656698    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:05.754754 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:05.754705    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:05.916618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:05.916531    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:06.720006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:06.719961    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:07.521006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:07.520972    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:08.147928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:08.147889    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:08.248142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:08.248111    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:08.635509 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:08.635474    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:08.735595 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:08.735562    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:08.812101 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:08.812066    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:26.096225 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:26.096194    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:30.295632 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:30.295592    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:30.900408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:30.900373    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:31.699214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:31.699171    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:32.323544 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:32.323508    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:33.007305 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:33.007273    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:33.706345 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:33.706314    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:34.311278 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:34.311237    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:35.020508 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:35.020462    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:36.938954 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:36.938922    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:37.496798 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:37.496761    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:37.575623 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:37.575534    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:37.646689 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:37.646657    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:37.744066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:37.744036    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:38.130382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:38.130351    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:38.861177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:38.861139    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:38.929645 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:38.929614    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:39.387296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:39.387262    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:39.642326 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:39.642224    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:39.711654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:39.711620    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:40.049049 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:40.049012    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:40.419884 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:40.419783    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:41.303471 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:41.303433    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:41.941401 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:41.941363    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:42.026308 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:42.026270    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:42.087369 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:42.087331    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:42.915555 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:42.915524    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:43.629922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:43.629849    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:44.528143 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:44.528087    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:44.594171 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:44.594136    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:44.821017 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:44.820929    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:44.966111 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:44.966076    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:45.056966 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:45.056923    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:48.393325 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:48.393240    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:57.087414 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:57.087377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:57.223535 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:57.223497    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:57.831763 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:57.831720    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:58.540806 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:58.540771    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:59.173481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:59.173442    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:59.814432 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:59.814389    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:54:59.997557 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:54:59.997525    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:00.397660 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:00.397623    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:00.618549 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:00.618519    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:01.266437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:01.266399    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:01.358613 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:01.358572    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:02.046468 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:02.046427    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:02.751045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:02.751013    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:03.569836 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:03.569745    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:04.330416 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:04.330377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:05.116645 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:05.116610    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.224315 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.224281    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.731960 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.731923    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:55:07.732184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.732159    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" podUID="1c033029-c403-4755-a756-f433ad9fa80c" containerName="maas-api" containerID="cri-o://18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698" gracePeriod=30
Sep 11 03:55:07.760164 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.760134    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.823883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.823843    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.884808 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.884765    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.945332 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.945282    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:07.983752 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:07.983697    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:55:08.048957 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.048925    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca\") pod \"1c033029-c403-4755-a756-f433ad9fa80c\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") "
Sep 11 03:55:08.049159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.048980    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mxq9\" (UniqueName: \"kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9\") pod \"1c033029-c403-4755-a756-f433ad9fa80c\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") "
Sep 11 03:55:08.049159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.049056    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs\") pod \"1c033029-c403-4755-a756-f433ad9fa80c\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") "
Sep 11 03:55:08.049159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.049106    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") pod \"1c033029-c403-4755-a756-f433ad9fa80c\" (UID: \"1c033029-c403-4755-a756-f433ad9fa80c\") "
Sep 11 03:55:08.049391 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.049323    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c033029-c403-4755-a756-f433ad9fa80c" (UID: "1c033029-c403-4755-a756-f433ad9fa80c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:55:08.049730 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.049702    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c033029-c403-4755-a756-f433ad9fa80c-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:55:08.051423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.051401    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c033029-c403-4755-a756-f433ad9fa80c" (UID: "1c033029-c403-4755-a756-f433ad9fa80c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:55:08.051515 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.051490    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9" (OuterVolumeSpecName: "kube-api-access-9mxq9") pod "1c033029-c403-4755-a756-f433ad9fa80c" (UID: "1c033029-c403-4755-a756-f433ad9fa80c"). InnerVolumeSpecName "kube-api-access-9mxq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:55:08.051558 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.051535    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c033029-c403-4755-a756-f433ad9fa80c" (UID: "1c033029-c403-4755-a756-f433ad9fa80c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:55:08.150915 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.150877    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:55:08.150915 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.150908    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c033029-c403-4755-a756-f433ad9fa80c-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:55:08.150915 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.150921    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mxq9\" (UniqueName: \"kubernetes.io/projected/1c033029-c403-4755-a756-f433ad9fa80c-kube-api-access-9mxq9\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:55:08.326611 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.326579    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:08.427930 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.427897    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:08.444755 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.444729    2553 generic.go:356] "Generic (PLEG): container finished" podID="1c033029-c403-4755-a756-f433ad9fa80c" containerID="18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698" exitCode=0
Sep 11 03:55:08.444907 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.444779    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"
Sep 11 03:55:08.444907 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.444829    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" event={"ID":"1c033029-c403-4755-a756-f433ad9fa80c","Type":"ContainerDied","Data":"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"}
Sep 11 03:55:08.444907 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.444891    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24" event={"ID":"1c033029-c403-4755-a756-f433ad9fa80c","Type":"ContainerDied","Data":"8c38820a087d7fb1165bf797c571b129e59655d35fd5cf3f668457d83c838522"}
Sep 11 03:55:08.445039 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.444922    2553 scope.go:117] "RemoveContainer" containerID="18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"
Sep 11 03:55:08.457092 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.457074    2553 scope.go:117] "RemoveContainer" containerID="18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"
Sep 11 03:55:08.457369 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:55:08.457350    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698\": container with ID starting with 18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698 not found: ID does not exist" containerID="18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"
Sep 11 03:55:08.457438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.457377    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698"} err="failed to get container status \"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698\": rpc error: code = NotFound desc = could not find container \"18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698\": container with ID starting with 18bc5862afa6c0926dc08530a22b42f1104e5a224f6ef1471b2ee834e265a698 not found: ID does not exist"
Sep 11 03:55:08.468451 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.468426    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:55:08.471510 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.471489    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-fd0def-64c5bd75c8-26q24"]
Sep 11 03:55:08.878184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:08.878147    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c033029-c403-4755-a756-f433ad9fa80c" path="/var/lib/kubelet/pods/1c033029-c403-4755-a756-f433ad9fa80c/volumes"
Sep 11 03:55:09.038368 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:09.038337    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:09.658883 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:09.658810    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:10.506059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:10.506025    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:10.855929 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:10.855900    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:55:10.858360 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:10.858338    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 03:55:11.792024 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:11.791985    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.307151 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.307110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.363789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.363745    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.464282 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.464237    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.550028 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.549999    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.674683 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.674591    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:12.913930 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:12.913895    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:13.409621 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:13.409593    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:13.509647 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:13.509616    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:13.731112 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:13.731035    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:14.357335 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:14.357300    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:14.983312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:14.983280    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:19.372547 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:19.372496    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:19.640612 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:19.640529    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:20.115315 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:20.115282    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:20.440998 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:20.440892    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:20.716323 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:20.716241    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:21.286517 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:21.286479    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:21.342143 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:21.342110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:21.405023 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:21.404980    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:21.505633 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:21.505602    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:21.608595 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:21.608517    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:22.004182 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:22.004091    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:23.143537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:23.143502    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:23.777265 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:23.777233    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:23.866911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:23.866877    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:24.615326 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:24.615284    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:25.344911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:25.344870    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:25.444762 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:25.444720    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:25.693313 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:25.693233    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:25.887087 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:25.887036    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:25.971713 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:25.971621    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:27.324504 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:27.324469    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:27.425749 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:27.425718    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:27.888877 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:27.888845    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:27.989593 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:27.989558    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:28.489702 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:28.489669    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:47.256460 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:47.256417    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:48.366170 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:48.366077    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:48.557464 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:48.557419    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:48.809125 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:48.809092    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:49.295601 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:49.295566    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:53.825730 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:53.825692    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:53.878426 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:53.878398    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:53.949991 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:53.949964    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:55:54.050197 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:55:54.050165    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:09.887406 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:09.887363    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:14.293011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:14.292975    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:15.974380 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:15.974345    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:16.029576 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:16.029536    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:16.102668 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:16.102634    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:16.203959 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:16.203924    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:16.707221 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:16.707182    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:17.500213 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:17.500177    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:18.210165 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:18.210128    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:18.914657 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:18.914627    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:19.503810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:19.503767    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:26.667946 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:26.667912    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.235652 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.235618    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.337337 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.337298    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.412433 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.412402    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.472529 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.472496    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.762800 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.762766    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:27.864745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:27.864707    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:28.423321 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:28.423286    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:29.107640 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:29.107609    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:32.860881 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:32.860844    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:33.232095 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:33.231978    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:33.291103 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:33.291076    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:33.357719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:33.357676    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:33.458981 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:33.458939    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:33.558183 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:33.558148    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:34.178516 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:34.178469    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:34.279686 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:34.279650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:34.379790 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:34.379756    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:35.621704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:35.621676    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:35.891796 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:35.891715    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:36.698565 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:36.698536    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:36.930271 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:36.930234    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:37.687113 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:37.687082    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:38.348979 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:38.348946    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:38.824569 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:38.824540    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:38.960845 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:38.960796    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:39.065487 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:39.065454    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:39.518839 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:39.518780    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:39.844694 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:39.844607    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:39.949198 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:39.949163    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:40.859080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:40.859041    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:41.565105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:41.565070    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:41.654584 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:41.654550    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:41.776235 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:41.776202    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:42.135521 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:42.135484    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:42.194463 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:42.194431    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:42.802320 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:42.802282    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.289545 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.289514    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:56:47.290035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.289793    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" podUID="639894f8-1b35-4dab-853a-5d80bde40795" containerName="maas-api" containerID="cri-o://a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578" gracePeriod=30
Sep 11 03:56:47.316313 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.316276    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.419667 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.419628    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.484366 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.484317    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.534524 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.534500    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:56:47.537117 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.537089    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.685733 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.685656    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") pod \"639894f8-1b35-4dab-853a-5d80bde40795\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") "
Sep 11 03:56:47.685733 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.685688    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs\") pod \"639894f8-1b35-4dab-853a-5d80bde40795\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") "
Sep 11 03:56:47.685733 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.685716    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca\") pod \"639894f8-1b35-4dab-853a-5d80bde40795\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") "
Sep 11 03:56:47.686046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.685765    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jckn\" (UniqueName: \"kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn\") pod \"639894f8-1b35-4dab-853a-5d80bde40795\" (UID: \"639894f8-1b35-4dab-853a-5d80bde40795\") "
Sep 11 03:56:47.686111 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.686071    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "639894f8-1b35-4dab-853a-5d80bde40795" (UID: "639894f8-1b35-4dab-853a-5d80bde40795"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:56:47.687782 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.687757    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "639894f8-1b35-4dab-853a-5d80bde40795" (UID: "639894f8-1b35-4dab-853a-5d80bde40795"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:56:47.687890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.687799    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn" (OuterVolumeSpecName: "kube-api-access-8jckn") pod "639894f8-1b35-4dab-853a-5d80bde40795" (UID: "639894f8-1b35-4dab-853a-5d80bde40795"). InnerVolumeSpecName "kube-api-access-8jckn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:56:47.687890 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.687865    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "639894f8-1b35-4dab-853a-5d80bde40795" (UID: "639894f8-1b35-4dab-853a-5d80bde40795"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:56:47.787388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.787352    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/639894f8-1b35-4dab-853a-5d80bde40795-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:47.787388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.787382    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jckn\" (UniqueName: \"kubernetes.io/projected/639894f8-1b35-4dab-853a-5d80bde40795-kube-api-access-8jckn\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:47.787388 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.787393    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:47.787618 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.787403    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/639894f8-1b35-4dab-853a-5d80bde40795-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:47.835459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.835425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:47.932949 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:47.932917    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:48.023926 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.023898    2553 generic.go:356] "Generic (PLEG): container finished" podID="639894f8-1b35-4dab-853a-5d80bde40795" containerID="a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578" exitCode=0
Sep 11 03:56:48.024086 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.023953    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"
Sep 11 03:56:48.024086 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.023979    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" event={"ID":"639894f8-1b35-4dab-853a-5d80bde40795","Type":"ContainerDied","Data":"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"}
Sep 11 03:56:48.024086 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.024036    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb" event={"ID":"639894f8-1b35-4dab-853a-5d80bde40795","Type":"ContainerDied","Data":"507c9fcf4fa94d843c6aeb3df552f05ca22659ffd8a81179bf398b1d730d1f89"}
Sep 11 03:56:48.024086 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.024061    2553 scope.go:117] "RemoveContainer" containerID="a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"
Sep 11 03:56:48.033503 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.033476    2553 scope.go:117] "RemoveContainer" containerID="a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"
Sep 11 03:56:48.033843 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:56:48.033797    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578\": container with ID starting with a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578 not found: ID does not exist" containerID="a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"
Sep 11 03:56:48.033941 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.033857    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578"} err="failed to get container status \"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578\": rpc error: code = NotFound desc = could not find container \"a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578\": container with ID starting with a7035b602dfac49aff9633fbce5d5f39a9cd3c597bdf7538aca65e486c9e5578 not found: ID does not exist"
Sep 11 03:56:48.049020 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.048997    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:56:48.051871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.051851    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4c62f1-6f9589b887-w72gb"]
Sep 11 03:56:48.587273 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.587237    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:48.873105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:48.873023    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639894f8-1b35-4dab-853a-5d80bde40795" path="/var/lib/kubelet/pods/639894f8-1b35-4dab-853a-5d80bde40795/volumes"
Sep 11 03:56:49.111244 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:49.111201    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:49.214142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:49.214059    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:49.342592 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:49.342558    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:49.668717 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:49.668675    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:49.828097 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:49.828065    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:52.326670 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:52.326618    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:52.501628 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:52.501592    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:52.600437 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:52.600348    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:53.832041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832008    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:56:53.832486 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832468    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832486 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832486    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832499    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639894f8-1b35-4dab-853a-5d80bde40795" containerName="maas-api"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832505    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="639894f8-1b35-4dab-853a-5d80bde40795" containerName="maas-api"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832511    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c033029-c403-4755-a756-f433ad9fa80c" containerName="maas-api"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832517    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c033029-c403-4755-a756-f433ad9fa80c" containerName="maas-api"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832530    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832535    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832751 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832596    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832751 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832605    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa1a9cc-437d-4498-be61-917bdd15cdff" containerName="istio-proxy"
Sep 11 03:56:53.832751 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832613    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="639894f8-1b35-4dab-853a-5d80bde40795" containerName="maas-api"
Sep 11 03:56:53.832751 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.832618    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c033029-c403-4755-a756-f433ad9fa80c" containerName="maas-api"
Sep 11 03:56:53.836963 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.836941    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.839766 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.839731    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4f6c0189-serving-cert\""
Sep 11 03:56:53.843794 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.843761    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2jkq\" (UniqueName: \"kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.843932 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.843856    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.843932 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.843889    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.843932 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.843912    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.847423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.847404    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:56:53.853454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.853429    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:53.910325 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.910288    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:53.944430 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.944399    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2jkq\" (UniqueName: \"kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.944609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.944453    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.944609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.944478    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.944609 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.944494    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.944609 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:56:53.944596    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-serving-cert: secret "maas-api-e2e-mt-4f6c0189-serving-cert" not found
Sep 11 03:56:53.944793 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:56:53.944661    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls podName:aaa1619f-9499-4936-a5c1-248091ec1384 nodeName:}" failed. No retries permitted until 2026-09-11 03:56:54.444643973 +0000 UTC m=+1004.087434136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls") pod "maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" (UID: "aaa1619f-9499-4936-a5c1-248091ec1384") : secret "maas-api-e2e-mt-4f6c0189-serving-cert" not found
Sep 11 03:56:53.945163 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.945143    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.946769 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.946753    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:53.952453 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:53.952429    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2jkq\" (UniqueName: \"kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:54.010174 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.010139    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:54.111176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.111090    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:54.234852 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.234801    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:56:54.235048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.235022    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy" containerID="cri-o://9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e" gracePeriod=30
Sep 11 03:56:54.259113 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.259080    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:54.354642 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.354613    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:54.427340 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.427255    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:54.449285 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.449252    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:54.451607 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.451580    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") pod \"maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:54.748787 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:54.748699    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:55.024597 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:55.024568    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:56:55.025910 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:56:55.025873    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa1619f_9499_4936_a5c1_248091ec1384.slice/crio-ac48ee581e02cc90b09dbdaaa2d30ffb0e9c6606eec9cccc668fe0c91bb8eb0f WatchSource:0}: Error finding container ac48ee581e02cc90b09dbdaaa2d30ffb0e9c6606eec9cccc668fe0c91bb8eb0f: Status 404 returned error can't find the container with id ac48ee581e02cc90b09dbdaaa2d30ffb0e9c6606eec9cccc668fe0c91bb8eb0f
Sep 11 03:56:55.027214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:55.027190    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 03:56:55.064243 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:55.064200    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" event={"ID":"aaa1619f-9499-4936-a5c1-248091ec1384","Type":"ContainerStarted","Data":"ac48ee581e02cc90b09dbdaaa2d30ffb0e9c6606eec9cccc668fe0c91bb8eb0f"}
Sep 11 03:56:55.286523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:55.286443    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:55.935096 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:55.935051    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:56.071199 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:56.071161    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" event={"ID":"aaa1619f-9499-4936-a5c1-248091ec1384","Type":"ContainerStarted","Data":"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"}
Sep 11 03:56:56.071596 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:56.071251    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:56:56.088213 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:56.088170    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" podStartSLOduration=2.719321795 podStartE2EDuration="3.088157495s" totalImagesPullingTime="368.8357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:56:56.087865152 +0000 UTC m=+1005.730655315" watchObservedRunningTime="2026-09-11 03:56:56.088157495 +0000 UTC m=+1005.730947656"
Sep 11 03:56:56.638408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:56.638377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:56:59.494745 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.494716    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:56:59.599200 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599163    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599212    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599252    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599285    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599373 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599362    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599393    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599414    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599453    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k85jl\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599482    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy\") pod \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\" (UID: \"2693b79f-4964-42f7-a46c-553d7ef3a3c1\") "
Sep 11 03:56:59.599581 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599563    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:56:59.599871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599575    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:56:59.599871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599608    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:56:59.599871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599702    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:56:59.599871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599851    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:56:59.600043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599953    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.600043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599974    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.600043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.599987    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.600043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.600000    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.600043 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.600013    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.601620 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.601594    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 03:56:59.601761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.601628    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl" (OuterVolumeSpecName: "kube-api-access-k85jl") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "kube-api-access-k85jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:56:59.602047 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.602024    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 03:56:59.602097 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.602032    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "2693b79f-4964-42f7-a46c-553d7ef3a3c1" (UID: "2693b79f-4964-42f7-a46c-553d7ef3a3c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:56:59.700938 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.700905    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.700938 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.700932    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.700938 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.700941    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k85jl\" (UniqueName: \"kubernetes.io/projected/2693b79f-4964-42f7-a46c-553d7ef3a3c1-kube-api-access-k85jl\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:56:59.701174 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:56:59.700951    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2693b79f-4964-42f7-a46c-553d7ef3a3c1-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:00.094676 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.094638    2553 generic.go:356] "Generic (PLEG): container finished" podID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerID="9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e" exitCode=0
Sep 11 03:57:00.094861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.094703    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"
Sep 11 03:57:00.094861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.094726    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerDied","Data":"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"}
Sep 11 03:57:00.094861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.094789    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226" event={"ID":"2693b79f-4964-42f7-a46c-553d7ef3a3c1","Type":"ContainerDied","Data":"820ce6b8e5e5552e399b308c5b0efe9ffd8da2ccf449f26ac482bcf5adcff9e1"}
Sep 11 03:57:00.094861 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.094826    2553 scope.go:117] "RemoveContainer" containerID="9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"
Sep 11 03:57:00.104205 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.104187    2553 scope.go:117] "RemoveContainer" containerID="96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"
Sep 11 03:57:00.112355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.112336    2553 scope.go:117] "RemoveContainer" containerID="9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"
Sep 11 03:57:00.112620 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:00.112600    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e\": container with ID starting with 9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e not found: ID does not exist" containerID="9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"
Sep 11 03:57:00.112693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.112632    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e"} err="failed to get container status \"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e\": rpc error: code = NotFound desc = could not find container \"9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e\": container with ID starting with 9b8715a39e34e9fe5fa37ead44394ba86c88814839d643911c0925f8dca9ef2e not found: ID does not exist"
Sep 11 03:57:00.112693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.112657    2553 scope.go:117] "RemoveContainer" containerID="96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"
Sep 11 03:57:00.112956 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:00.112937    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a\": container with ID starting with 96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a not found: ID does not exist" containerID="96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"
Sep 11 03:57:00.113001 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.112963    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a"} err="failed to get container status \"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a\": rpc error: code = NotFound desc = could not find container \"96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a\": container with ID starting with 96bf97568938b5cfe88aa90b92257bc8ccda2dc30de933a1d8dcd8a62ab6ce6a not found: ID does not exist"
Sep 11 03:57:00.116429 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.116410    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:57:00.119959 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.119939    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4c62f1-openshift-default-d78d776b-ql226"]
Sep 11 03:57:00.872981 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:00.872947    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" path="/var/lib/kubelet/pods/2693b79f-4964-42f7-a46c-553d7ef3a3c1/volumes"
Sep 11 03:57:02.081525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:02.081492    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:57:04.791419 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:04.791385    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:05.282424 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.282388    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:57:05.282615 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.282594    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" podUID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" containerName="maas-api" containerID="cri-o://0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768" gracePeriod=30
Sep 11 03:57:05.302497 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.302465    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:05.401176 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.401143    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:05.474184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.474152    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:05.525914 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.525870    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:05.537987 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.537926    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:57:05.661180 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.661145    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvbd6\" (UniqueName: \"kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6\") pod \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") "
Sep 11 03:57:05.661390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.661205    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") pod \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") "
Sep 11 03:57:05.661390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.661270    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca\") pod \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") "
Sep 11 03:57:05.661390 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.661318    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs\") pod \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\" (UID: \"6c80d8ef-0aaa-4d6a-acc4-8012ad384218\") "
Sep 11 03:57:05.662223 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.662082    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c80d8ef-0aaa-4d6a-acc4-8012ad384218" (UID: "6c80d8ef-0aaa-4d6a-acc4-8012ad384218"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:57:05.663760 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.663737    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c80d8ef-0aaa-4d6a-acc4-8012ad384218" (UID: "6c80d8ef-0aaa-4d6a-acc4-8012ad384218"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:57:05.664153 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.664134    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6c80d8ef-0aaa-4d6a-acc4-8012ad384218" (UID: "6c80d8ef-0aaa-4d6a-acc4-8012ad384218"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:57:05.664220 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.664162    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6" (OuterVolumeSpecName: "kube-api-access-tvbd6") pod "6c80d8ef-0aaa-4d6a-acc4-8012ad384218" (UID: "6c80d8ef-0aaa-4d6a-acc4-8012ad384218"). InnerVolumeSpecName "kube-api-access-tvbd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:57:05.762526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.762495    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:05.762526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.762522    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvbd6\" (UniqueName: \"kubernetes.io/projected/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-kube-api-access-tvbd6\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:05.762526 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.762533    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:05.762749 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.762544    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c80d8ef-0aaa-4d6a-acc4-8012ad384218-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:05.908697 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:05.908569    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:06.005129 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.005092    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:06.132714 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.132678    2553 generic.go:356] "Generic (PLEG): container finished" podID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" containerID="0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768" exitCode=0
Sep 11 03:57:06.132874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.132716    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" event={"ID":"6c80d8ef-0aaa-4d6a-acc4-8012ad384218","Type":"ContainerDied","Data":"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"}
Sep 11 03:57:06.132874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.132774    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h" event={"ID":"6c80d8ef-0aaa-4d6a-acc4-8012ad384218","Type":"ContainerDied","Data":"ff2656be6859926e6028d386f10107997c45b20abf3e3817eec2890b1031489d"}
Sep 11 03:57:06.132874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.132797    2553 scope.go:117] "RemoveContainer" containerID="0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"
Sep 11 03:57:06.132874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.132728    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"
Sep 11 03:57:06.142580 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.142559    2553 scope.go:117] "RemoveContainer" containerID="0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"
Sep 11 03:57:06.142909 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:06.142886    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768\": container with ID starting with 0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768 not found: ID does not exist" containerID="0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"
Sep 11 03:57:06.142993 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.142919    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768"} err="failed to get container status \"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768\": rpc error: code = NotFound desc = could not find container \"0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768\": container with ID starting with 0d9826a13efaa3f265b95ce2c736f69168d205b75707a4ef0341d36ebd839768 not found: ID does not exist"
Sep 11 03:57:06.155761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.155740    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:57:06.159156 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.159105    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8efac4-9c4c74497-cxz9h"]
Sep 11 03:57:06.716734 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.716695    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:06.872931 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:06.872897    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" path="/var/lib/kubelet/pods/6c80d8ef-0aaa-4d6a-acc4-8012ad384218/volumes"
Sep 11 03:57:08.022175 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:08.022125    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:08.724859 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:08.724805    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:11.756523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:11.756479    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:11.859447 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:11.859410    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:11.957664 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:11.957629    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:12.021083 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.021055    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:12.368385 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.368294    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:12.586461 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.586425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:12.848381 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.848349    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 03:57:12.849303 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849279    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" containerName="maas-api"
Sep 11 03:57:12.849303 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849303    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" containerName="maas-api"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849333    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849342    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849360    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849367    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849469    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:12.849482 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.849480    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c80d8ef-0aaa-4d6a-acc4-8012ad384218" containerName="maas-api"
Sep 11 03:57:12.853717 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.853695    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.856928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.856906    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b4d3cf-openshift-default-dockercfg-dw5b8\""
Sep 11 03:57:12.864340 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.864305    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 03:57:12.880064 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.880044    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:12.929873 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929842    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.929980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929888    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.929980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929920    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.929980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929943    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtt8c\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.929980 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929964    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.930135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.929991    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.930135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.930042    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.930135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.930080    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:12.930135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:12.930117    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.024650 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.024608    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:13.030720 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.030695    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.030847 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.030748    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.030926 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.030869    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.030986 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.030966    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.030990    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtt8c\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031015    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031140 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031054    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031140 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031090    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031140 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031133    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031508 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031460    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031524    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031529    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031684 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031614    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.031684 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.031624    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.033335 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.033304    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.033557 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.033536    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.052030 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.052006    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.052204 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.052180    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtt8c\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c\") pod \"e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") " pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.145310 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.145239    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:13.166557 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.166526    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:13.236294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.236242    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:13.449533 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.449507    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 03:57:13.451962 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:57:13.451935    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3cb689_b5ae_4dfe_8c5a_cad81d5c15d2.slice/crio-2ee14a5f418fe2a68187b712ba8ecf0ebc750c7f8f139bc3d3749b4029fe7314 WatchSource:0}: Error finding container 2ee14a5f418fe2a68187b712ba8ecf0ebc750c7f8f139bc3d3749b4029fe7314: Status 404 returned error can't find the container with id 2ee14a5f418fe2a68187b712ba8ecf0ebc750c7f8f139bc3d3749b4029fe7314
Sep 11 03:57:13.454068 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.454040    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:13.454135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.454102    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:13.454135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.454130    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:13.495293 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:13.495263    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:14.176582 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:14.176548    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerStarted","Data":"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"}
Sep 11 03:57:14.177022 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:14.176601    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerStarted","Data":"2ee14a5f418fe2a68187b712ba8ecf0ebc750c7f8f139bc3d3749b4029fe7314"}
Sep 11 03:57:14.198133 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:14.198092    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podStartSLOduration=2.198077346 podStartE2EDuration="2.198077346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:57:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:57:14.197330998 +0000 UTC m=+1023.840121185" watchObservedRunningTime="2026-09-11 03:57:14.198077346 +0000 UTC m=+1023.840867578"
Sep 11 03:57:15.167642 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:15.167605    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:15.173680 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:15.173649    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:15.182042 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:15.182008    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:15.182724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:15.182704    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:15.416367 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:15.416331    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:16.109530 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:16.109500    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:16.204616 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:16.204584    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.231438 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.231408    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 03:57:17.231927 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.231910    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2693b79f-4964-42f7-a46c-553d7ef3a3c1" containerName="istio-proxy"
Sep 11 03:57:17.234656 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.234633    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.237280 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.237254    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-b4d3cf-serving-cert\""
Sep 11 03:57:17.247682 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.247646    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 03:57:17.265159 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.265134    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.365479 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.365448    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.371477 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.371449    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kzmx\" (UniqueName: \"kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.371591 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.371479    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.371591 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.371576    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.371676 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.371622    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.472619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.472585    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kzmx\" (UniqueName: \"kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.472619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.472620    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.472783 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.472679    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.472783 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.472704    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.472889 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:17.472802    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-serving-cert: secret "maas-api-e2e-shared-a-w3-b4d3cf-serving-cert" not found
Sep 11 03:57:17.472934 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:17.472896    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls podName:291cbbeb-83ae-4b59-84bc-569f9a4cc98b nodeName:}" failed. No retries permitted until 2026-09-11 03:57:17.972876555 +0000 UTC m=+1027.615666694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls") pod "maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" (UID: "291cbbeb-83ae-4b59-84bc-569f9a4cc98b") : secret "maas-api-e2e-shared-a-w3-b4d3cf-serving-cert" not found
Sep 11 03:57:17.473408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.473389    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.475127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.475108    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.485315 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.485259    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kzmx\" (UniqueName: \"kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.745761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.745650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.848504 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.848470    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.945491 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.945453    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:17.978508 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.978480    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:17.981184 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:17.981160    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:18.149893 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:18.149860    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:18.428375 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:18.428351    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 03:57:18.429635 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:57:18.429606    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291cbbeb_83ae_4b59_84bc_569f9a4cc98b.slice/crio-3387f24e541082a65979e58c7f0eae33bc940b0de261a40d438c6c21c23595ae WatchSource:0}: Error finding container 3387f24e541082a65979e58c7f0eae33bc940b0de261a40d438c6c21c23595ae: Status 404 returned error can't find the container with id 3387f24e541082a65979e58c7f0eae33bc940b0de261a40d438c6c21c23595ae
Sep 11 03:57:19.205874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.205766    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" event={"ID":"291cbbeb-83ae-4b59-84bc-569f9a4cc98b","Type":"ContainerStarted","Data":"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"}
Sep 11 03:57:19.205874 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.205839    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" event={"ID":"291cbbeb-83ae-4b59-84bc-569f9a4cc98b","Type":"ContainerStarted","Data":"3387f24e541082a65979e58c7f0eae33bc940b0de261a40d438c6c21c23595ae"}
Sep 11 03:57:19.206059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.205966    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:19.224069 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.224019    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" podStartSLOduration=1.886512301 podStartE2EDuration="2.224004424s" totalImagesPullingTime="337.492123ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:57:19.222232003 +0000 UTC m=+1028.865022166" watchObservedRunningTime="2026-09-11 03:57:19.224004424 +0000 UTC m=+1028.866794585"
Sep 11 03:57:19.266571 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.266546    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:19.322639 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.322611    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:19.735655 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:19.735620    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:20.611404 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:20.611371    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:21.424339 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:21.424308    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:22.019744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:22.019710    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:22.570588 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:22.570547    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:22.621872 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:22.621835    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.220525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.220430    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:57:24.221008 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.220683    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" podUID="aaa1619f-9499-4936-a5c1-248091ec1384" containerName="maas-api" containerID="cri-o://edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b" gracePeriod=30
Sep 11 03:57:24.248800 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.248731    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.324478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.324443    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.424107 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.424049    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.477288 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.477162    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.524758 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.524724    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:57:24.652964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.652119    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") pod \"aaa1619f-9499-4936-a5c1-248091ec1384\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") "
Sep 11 03:57:24.652964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.652201    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca\") pod \"aaa1619f-9499-4936-a5c1-248091ec1384\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") "
Sep 11 03:57:24.652964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.652256    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs\") pod \"aaa1619f-9499-4936-a5c1-248091ec1384\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") "
Sep 11 03:57:24.652964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.652333    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2jkq\" (UniqueName: \"kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq\") pod \"aaa1619f-9499-4936-a5c1-248091ec1384\" (UID: \"aaa1619f-9499-4936-a5c1-248091ec1384\") "
Sep 11 03:57:24.653359 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.653118    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aaa1619f-9499-4936-a5c1-248091ec1384" (UID: "aaa1619f-9499-4936-a5c1-248091ec1384"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 03:57:24.667522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.667274    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq" (OuterVolumeSpecName: "kube-api-access-l2jkq") pod "aaa1619f-9499-4936-a5c1-248091ec1384" (UID: "aaa1619f-9499-4936-a5c1-248091ec1384"). InnerVolumeSpecName "kube-api-access-l2jkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:57:24.668318 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.668272    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aaa1619f-9499-4936-a5c1-248091ec1384" (UID: "aaa1619f-9499-4936-a5c1-248091ec1384"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:57:24.668718 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.668682    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aaa1619f-9499-4936-a5c1-248091ec1384" (UID: "aaa1619f-9499-4936-a5c1-248091ec1384"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 03:57:24.753854 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.753690    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2jkq\" (UniqueName: \"kubernetes.io/projected/aaa1619f-9499-4936-a5c1-248091ec1384-kube-api-access-l2jkq\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:24.753854 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.753725    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:24.753854 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.753739    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa1619f-9499-4936-a5c1-248091ec1384-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:24.753854 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.753754    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aaa1619f-9499-4936-a5c1-248091ec1384-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:57:24.852879 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.852846    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:24.953619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:24.953567    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:25.216704 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.216679    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 03:57:25.242551 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.242511    2553 generic.go:356] "Generic (PLEG): container finished" podID="aaa1619f-9499-4936-a5c1-248091ec1384" containerID="edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b" exitCode=0
Sep 11 03:57:25.243040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.242699    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" event={"ID":"aaa1619f-9499-4936-a5c1-248091ec1384","Type":"ContainerDied","Data":"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"}
Sep 11 03:57:25.243040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.242751    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q" event={"ID":"aaa1619f-9499-4936-a5c1-248091ec1384","Type":"ContainerDied","Data":"ac48ee581e02cc90b09dbdaaa2d30ffb0e9c6606eec9cccc668fe0c91bb8eb0f"}
Sep 11 03:57:25.243040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.242784    2553 scope.go:117] "RemoveContainer" containerID="edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"
Sep 11 03:57:25.243040 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.242979    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"
Sep 11 03:57:25.262782 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.262761    2553 scope.go:117] "RemoveContainer" containerID="edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"
Sep 11 03:57:25.263341 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:57:25.263315    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b\": container with ID starting with edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b not found: ID does not exist" containerID="edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"
Sep 11 03:57:25.263530 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.263504    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b"} err="failed to get container status \"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b\": rpc error: code = NotFound desc = could not find container \"edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b\": container with ID starting with edae6ee402fda2f6e56652f8d5d70c21496d57ecc201659d6a4963fbb76a090b not found: ID does not exist"
Sep 11 03:57:25.272885 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.272860    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:57:25.275293 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.275270    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4f6c0189-75b4bbdb7d-x4k7q"]
Sep 11 03:57:25.648600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:25.648571    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:26.874336 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:26.874294    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa1619f-9499-4936-a5c1-248091ec1384" path="/var/lib/kubelet/pods/aaa1619f-9499-4936-a5c1-248091ec1384/volumes"
Sep 11 03:57:27.020397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:27.020356    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:27.817391 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:27.817349    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:30.294045 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.294016    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:34890->10.132.0.59:15021: read: connection reset by peer" start-of-body=
Sep 11 03:57:30.294428 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.294097    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:34890->10.132.0.59:15021: read: connection reset by peer"
Sep 11 03:57:30.582552 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.582476    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:30.638444 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.638407    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:30.828575 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.828537    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:30.889651 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:30.889574    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:31.185948 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.185843    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:31.286496 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.286459    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:31.308219 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.308193    2553 generic.go:356] "Generic (PLEG): container finished" podID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerID="eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3" exitCode=0
Sep 11 03:57:31.308577 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.308236    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerDied","Data":"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"}
Sep 11 03:57:31.308631 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.308619    2553 scope.go:117] "RemoveContainer" containerID="eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"
Sep 11 03:57:31.309654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.309627    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:31.309712 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.309697    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:31.309752 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.309729    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:31.436291 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.436212    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:31.536929 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.536868    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:31.686196 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:31.686158    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:32.315542 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:32.315505    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerStarted","Data":"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"}
Sep 11 03:57:33.167619 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:33.167586    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:33.167846 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:33.167807    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:33.167906 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:33.167873    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:34.167650 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:34.167600    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:34.168151 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:34.167674    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:34.446508 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:34.446415    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:35.043919 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:35.043884    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:35.142422 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:35.142389    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:35.167289 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:35.167249    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:35.167460 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:35.167316    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:35.243058 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:35.243025    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:36.167936 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.167901    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:36.168105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.167958    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:36.272925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.272885    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:36.366530 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.365598    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:36.464713 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.464607    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:36.863644 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.863606    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:36.959429 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:36.959397    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:37.167634 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.167542    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:37.167634 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.167605    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:37.556560 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.556526    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:37.645988 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.645955    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:37.814199 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.814113    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:37.906597 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:37.906569    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:38.006746 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:38.006711    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:38.167982 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:38.167900    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Sep 11 03:57:38.167982 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:38.167955    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Sep 11 03:57:38.368653 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:38.368621    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:39.170895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:39.170863    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:39.171360 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:39.171244    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:39.171879 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:39.171856    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 03:57:43.547100 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:43.547068    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:43.606651 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:43.606617    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:43.829965 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:43.829866    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.068809 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.068770    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.185923 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.185834    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.522177 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.522141    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.650455 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.650418    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.775150 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.775073    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 03:57:44.775478 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.775466    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa1619f-9499-4936-a5c1-248091ec1384" containerName="maas-api"
Sep 11 03:57:44.775522 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.775480    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa1619f-9499-4936-a5c1-248091ec1384" containerName="maas-api"
Sep 11 03:57:44.775562 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.775553    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa1619f-9499-4936-a5c1-248091ec1384" containerName="maas-api"
Sep 11 03:57:44.777589 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.777567    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.780189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.780168    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-20ae9a-openshift-default-dockercfg-ftrfg\""
Sep 11 03:57:44.789854 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.789802    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 03:57:44.804893 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.804873    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.843105 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843077    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843112    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843214 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843141    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843208    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843238    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843411 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843374    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rcbr\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843465 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843433    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843468    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.843523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.843502    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.904470 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.904439    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:44.945110 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945077    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rcbr\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945144    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945184    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945217    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945264 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945256    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945282    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945638 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945310    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945789 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945758    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.945892 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.945685    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.946035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.946009    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.946158 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.946051    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.946428 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.946402    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.946514 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.946437    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.946692 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.946668    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.948330 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.948303    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.948521 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.948500    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.962382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.962348    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:44.962723 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:44.962683    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rcbr\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr\") pod \"e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") " pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:45.089047 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.088969    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:45.362284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.362247    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 03:57:45.363268 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:57:45.363241    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141039e4_ef92_4d6b_b587_15b436e03b1d.slice/crio-2c05fd663d370a2318cca427227f165c9cbf003f877e711cfeb17b00759fe0d8 WatchSource:0}: Error finding container 2c05fd663d370a2318cca427227f165c9cbf003f877e711cfeb17b00759fe0d8: Status 404 returned error can't find the container with id 2c05fd663d370a2318cca427227f165c9cbf003f877e711cfeb17b00759fe0d8
Sep 11 03:57:45.365557 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.365527    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:45.365654 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.365602    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:45.365718 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.365670    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:57:45.402629 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:45.402600    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerStarted","Data":"2c05fd663d370a2318cca427227f165c9cbf003f877e711cfeb17b00759fe0d8"}
Sep 11 03:57:46.411025 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:46.410990    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerStarted","Data":"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"}
Sep 11 03:57:46.429691 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:46.429646    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podStartSLOduration=2.429631849 podStartE2EDuration="2.429631849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:57:46.428188281 +0000 UTC m=+1056.070978448" watchObservedRunningTime="2026-09-11 03:57:46.429631849 +0000 UTC m=+1056.072422011"
Sep 11 03:57:47.089403 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:47.089369    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:47.094308 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:47.094287    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:47.417135 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:47.417044    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:47.417939 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:47.417918    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:57:48.186869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:48.186831    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:48.347141 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:48.347043    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:48.435181 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:48.435150    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:48.925500 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:48.925452    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:49.265151 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:49.265118    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:49.730974 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:49.730883    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:50.535494 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:50.535458    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:50.636358 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:50.636325    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:50.688887 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:50.688848    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:50.967237 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:50.967158    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:51.069466 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:51.069430    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:51.338072 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:51.338031    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:52.108375 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:52.108345    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:52.822409 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:52.822349    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:53.408468 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:53.408436    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:54.892459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:54.892420    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:57:57.253294 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:57:57.253261    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:07.086617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.086582    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:07.531936 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.531906    2553 generic.go:356] "Generic (PLEG): container finished" podID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerID="4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e" exitCode=0
Sep 11 03:58:07.532113 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.531960    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerDied","Data":"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"}
Sep 11 03:58:07.532334 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.532321    2553 scope.go:117] "RemoveContainer" containerID="4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"
Sep 11 03:58:07.533506 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.533454    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:07.533803 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.533543    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:07.533803 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:07.533575    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:08.546398 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:08.546353    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerStarted","Data":"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"}
Sep 11 03:58:09.089829 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:09.089791    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:58:09.090013 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:09.089994    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 03:58:09.090064 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:09.090039    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 03:58:10.089544 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:10.089505    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 03:58:10.090041 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:10.089571    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 03:58:11.089510 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:11.089475    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 03:58:11.089673 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:11.089533    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 03:58:11.361596 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:11.361544    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:12.089292 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:12.089250    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 03:58:12.089459 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:12.089318    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 03:58:12.726910 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:12.726868    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:13.090015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:13.089974    2553 patch_prober.go:28] interesting pod/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Sep 11 03:58:13.090191 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:13.090033    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Sep 11 03:58:13.344700 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:13.344613    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:13.635378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:13.635285    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:13.733430 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:13.733395    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:14.002966 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.002866    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:14.096673 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.096641    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:58:14.097042 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.097015    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:58:14.097719 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.097700    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 03:58:14.324544 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.324503    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:14.535715 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:14.535683    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:15.008006 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:15.007971    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:16.102910 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:16.102881    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:16.699422 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:16.699388    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:16.797149 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:16.797116    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:16.897425 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:16.897392    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:17.723189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:17.723145    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:18.435870 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:18.435836    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.011147 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.011114    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 03:58:19.013891 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.013873    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.016466 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.016443    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-7dc787-serving-cert\""
Sep 11 03:58:19.026472 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.026448    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 03:58:19.043109 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.043083    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.144136 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.144101    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.174152 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.174122    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz54f\" (UniqueName: \"kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.174296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.174177    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.174296 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.174260    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.174382 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.174304    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.275436 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.275349    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.275436 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.275392    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.275677 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.275478    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz54f\" (UniqueName: \"kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.275677 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.275515    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.275677 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:19.275559    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-serving-cert: secret "maas-api-e2e-shared-b-w2-7dc787-serving-cert" not found
Sep 11 03:58:19.275677 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:19.275628    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls podName:11d28784-cc4f-4e03-8b2b-b6350e44261f nodeName:}" failed. No retries permitted until 2026-09-11 03:58:19.775611786 +0000 UTC m=+1089.418401951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls") pod "maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" (UID: "11d28784-cc4f-4e03-8b2b-b6350e44261f") : secret "maas-api-e2e-shared-b-w2-7dc787-serving-cert" not found
Sep 11 03:58:19.276127 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.276103    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.277885 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.277866    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.284295 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.284273    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz54f\" (UniqueName: \"kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.398842 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.398781    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.502773 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.502729    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.577617 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.577546    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:19.781408 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.781367    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.783877 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.783849    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:19.927157 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:19.927081    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:20.197332 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:20.197306    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 03:58:20.198642 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:58:20.198611    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d28784_cc4f_4e03_8b2b_b6350e44261f.slice/crio-3e75403a689133d251f59bcbc5b9200666f496b8ac0fccbb26990d8a49cf1abd WatchSource:0}: Error finding container 3e75403a689133d251f59bcbc5b9200666f496b8ac0fccbb26990d8a49cf1abd: Status 404 returned error can't find the container with id 3e75403a689133d251f59bcbc5b9200666f496b8ac0fccbb26990d8a49cf1abd
Sep 11 03:58:20.619181 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:20.619142    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" event={"ID":"11d28784-cc4f-4e03-8b2b-b6350e44261f","Type":"ContainerStarted","Data":"3e75403a689133d251f59bcbc5b9200666f496b8ac0fccbb26990d8a49cf1abd"}
Sep 11 03:58:21.626057 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:21.626021    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" event={"ID":"11d28784-cc4f-4e03-8b2b-b6350e44261f","Type":"ContainerStarted","Data":"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"}
Sep 11 03:58:21.626418 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:21.626082    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:21.643120 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:21.643072    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" podStartSLOduration=3.201813673 podStartE2EDuration="3.643054641s" totalImagesPullingTime="441.240968ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:58:21.641797578 +0000 UTC m=+1091.284587753" watchObservedRunningTime="2026-09-11 03:58:21.643054641 +0000 UTC m=+1091.285844803"
Sep 11 03:58:26.040285 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:26.040232    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:27.636693 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:27.636660    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 03:58:41.628423 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:41.628385    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:41.729297 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:41.729263    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:42.339454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:42.339425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:42.957210 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:42.957165    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:43.554460 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:43.554425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:43.654606 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:43.654570    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:43.906428 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:43.906344    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:44.157971 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.157882    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:44.205397 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.205364    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-e8997b"]
Sep 11 03:58:44.208869 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.208844    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:44.211355 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.211329    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fjqk5\""
Sep 11 03:58:44.212565 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.212544    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-e8997b"]
Sep 11 03:58:44.327055 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.327023    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6mk2\" (UniqueName: \"kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2\") pod \"test-curl-7701-e8997b\" (UID: \"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2\") " pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:44.427857 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.427753    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6mk2\" (UniqueName: \"kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2\") pod \"test-curl-7701-e8997b\" (UID: \"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2\") " pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:44.435409 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.435387    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6mk2\" (UniqueName: \"kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2\") pod \"test-curl-7701-e8997b\" (UID: \"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2\") " pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:44.519613 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.519579    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:44.790365 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:44.790332    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-e8997b"]
Sep 11 03:58:44.791488 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:58:44.791461    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05afa6e3_ca7f_4f0a_b9b3_ee3ef31009c2.slice/crio-85c2550574ba957eeec43f1c0e56a8568a660838ed5b16d2867f5346b122f4af WatchSource:0}: Error finding container 85c2550574ba957eeec43f1c0e56a8568a660838ed5b16d2867f5346b122f4af: Status 404 returned error can't find the container with id 85c2550574ba957eeec43f1c0e56a8568a660838ed5b16d2867f5346b122f4af
Sep 11 03:58:45.772217 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:45.772181    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-e8997b" event={"ID":"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2","Type":"ContainerStarted","Data":"73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198"}
Sep 11 03:58:45.772685 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:45.772225    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-e8997b" event={"ID":"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2","Type":"ContainerStarted","Data":"85c2550574ba957eeec43f1c0e56a8568a660838ed5b16d2867f5346b122f4af"}
Sep 11 03:58:45.787777 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:45.787728    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7701-e8997b" podStartSLOduration=1.137127388 podStartE2EDuration="1.787714006s" totalImagesPullingTime="650.586618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:58:45.785633487 +0000 UTC m=+1115.428423650" watchObservedRunningTime="2026-09-11 03:58:45.787714006 +0000 UTC m=+1115.430504180"
Sep 11 03:58:45.795035 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:45.795007    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:45.841683 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:45.841659    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:46.370863 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:46.370633    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-e8997b"]
Sep 11 03:58:46.374306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:46.374281    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-e8997b"]
Sep 11 03:58:46.777744 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:46.777712    2553 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7701-e8997b" container="test-curl-7701-e8997b"
Sep 11 03:58:47.783284 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:47.783244    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7701-e8997b" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" containerName="test-curl-7701-e8997b" containerID="cri-o://73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198" gracePeriod=2
Sep 11 03:58:48.213537 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.213461    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7701-1aa382"]
Sep 11 03:58:48.213886 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.213870    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" containerName="test-curl-7701-e8997b"
Sep 11 03:58:48.213933 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.213888    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" containerName="test-curl-7701-e8997b"
Sep 11 03:58:48.213969 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.213961    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" containerName="test-curl-7701-e8997b"
Sep 11 03:58:48.217168 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.217146    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:48.220314 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.220289    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-1aa382"]
Sep 11 03:58:48.361011 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.360975    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvcqz\" (UniqueName: \"kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz\") pod \"test-curl-7701-1aa382\" (UID: \"53b71449-d721-4323-8006-c52244c5a54b\") " pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:48.461681 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.461647    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvcqz\" (UniqueName: \"kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz\") pod \"test-curl-7701-1aa382\" (UID: \"53b71449-d721-4323-8006-c52244c5a54b\") " pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:48.469398 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.469343    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvcqz\" (UniqueName: \"kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz\") pod \"test-curl-7701-1aa382\" (UID: \"53b71449-d721-4323-8006-c52244c5a54b\") " pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:48.528083 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.528057    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:48.815563 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:48.815534    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7701-1aa382"]
Sep 11 03:58:48.817762 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:58:48.817731    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53b71449_d721_4323_8006_c52244c5a54b.slice/crio-13ea6f818fe3d40fcf8c6f3d993042fb800aa90c810ac1bd0523dc6e99fee033 WatchSource:0}: Error finding container 13ea6f818fe3d40fcf8c6f3d993042fb800aa90c810ac1bd0523dc6e99fee033: Status 404 returned error can't find the container with id 13ea6f818fe3d40fcf8c6f3d993042fb800aa90c810ac1bd0523dc6e99fee033
Sep 11 03:58:49.797080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:49.796939    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-1aa382" event={"ID":"53b71449-d721-4323-8006-c52244c5a54b","Type":"ContainerStarted","Data":"ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1"}
Sep 11 03:58:49.797080 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:49.796987    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7701-1aa382" event={"ID":"53b71449-d721-4323-8006-c52244c5a54b","Type":"ContainerStarted","Data":"13ea6f818fe3d40fcf8c6f3d993042fb800aa90c810ac1bd0523dc6e99fee033"}
Sep 11 03:58:49.811271 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:49.811224    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7701-1aa382" podStartSLOduration=1.811209352 podStartE2EDuration="1.811209352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:58:49.809569233 +0000 UTC m=+1119.452359395" watchObservedRunningTime="2026-09-11 03:58:49.811209352 +0000 UTC m=+1119.453999513"
Sep 11 03:58:49.972570 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:49.972536    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:50.080378 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.080283    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6mk2\" (UniqueName: \"kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2\") pod \"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2\" (UID: \"05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2\") "
Sep 11 03:58:50.082414 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.082385    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2" (OuterVolumeSpecName: "kube-api-access-l6mk2") pod "05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" (UID: "05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2"). InnerVolumeSpecName "kube-api-access-l6mk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:58:50.181471 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.181445    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6mk2\" (UniqueName: \"kubernetes.io/projected/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2-kube-api-access-l6mk2\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:58:50.372831 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.372723    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7701-1aa382"]
Sep 11 03:58:50.376015 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.375984    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7701-1aa382"]
Sep 11 03:58:50.803847 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.803788    2553 generic.go:356] "Generic (PLEG): container finished" podID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" containerID="73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198" exitCode=137
Sep 11 03:58:50.804046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.803856    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:50.804046 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.803892    2553 scope.go:117] "RemoveContainer" containerID="73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198"
Sep 11 03:58:50.804161 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:50.804119    2553 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7701-1aa382" container="test-curl-7701-1aa382"
Sep 11 03:58:50.817388 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:50.817355    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7701-e8997b\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:50.817681 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.817662    2553 scope.go:117] "RemoveContainer" containerID="73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198"
Sep 11 03:58:50.818011 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:50.817992    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198\": container with ID starting with 73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198 not found: ID does not exist" containerID="73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198"
Sep 11 03:58:50.818067 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.818021    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198"} err="failed to get container status \"73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198\": rpc error: code = NotFound desc = could not find container \"73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198\": container with ID starting with 73b519bdc28410c2f4ebc75a68539b6f46512c7669f4db37d315c7b0f603e198 not found: ID does not exist"
Sep 11 03:58:50.872066 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:50.872025    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7701-e8997b\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" pod="opendatahub/test-curl-7701-e8997b"
Sep 11 03:58:50.873433 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.873411    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2" path="/var/lib/kubelet/pods/05afa6e3-ca7f-4f0a-b9b3-ee3ef31009c2/volumes"
Sep 11 03:58:50.964845 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:50.964776    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:51.250078 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.249977    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 03:58:51.250598 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.250576    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b71449-d721-4323-8006-c52244c5a54b" containerName="test-curl-7701-1aa382"
Sep 11 03:58:51.250686 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.250603    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b71449-d721-4323-8006-c52244c5a54b" containerName="test-curl-7701-1aa382"
Sep 11 03:58:51.250760 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.250747    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b71449-d721-4323-8006-c52244c5a54b" containerName="test-curl-7701-1aa382"
Sep 11 03:58:51.255202 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.255179    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.257744 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.257723    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-89230689-openshift-default-dockercfg-5rtll\""
Sep 11 03:58:51.265905 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.265870    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 03:58:51.269319 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.269292    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:51.368312 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.368279    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:51.393451 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393419    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393465    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393495    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6hr\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393518    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393536    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393600 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393588    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393633    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393713    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.393908 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.393763    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494730 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494698    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494739    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494777    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494802    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494847    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6hr\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494869    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.494922 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.494892    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495018    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495208 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495057    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495311 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495199    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495311 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495239    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495311 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495299    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495460 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495406    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.495724 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.495705    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.497286 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.497266    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.497370 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.497286    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.502493 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.502434    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.502573 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.502520    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6hr\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr\") pod \"e2e-mt-89230689-openshift-default-554676cc96-nf7hw\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") " pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.567457 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.567410    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:51.811247 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.811204    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7701-1aa382" podUID="53b71449-d721-4323-8006-c52244c5a54b" containerName="test-curl-7701-1aa382" containerID="cri-o://ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1" gracePeriod=2
Sep 11 03:58:51.840189 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.840158    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 03:58:51.841757 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:58:51.841725    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16bf9f4_9622_4e6b_b22d_bdf90a4dd0be.slice/crio-1f5bbfe7cf7e6032f56be80e7a598a19a8b5d0786e533febffb36298c070a785 WatchSource:0}: Error finding container 1f5bbfe7cf7e6032f56be80e7a598a19a8b5d0786e533febffb36298c070a785: Status 404 returned error can't find the container with id 1f5bbfe7cf7e6032f56be80e7a598a19a8b5d0786e533febffb36298c070a785
Sep 11 03:58:51.844761 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.844721    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:51.844925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.844805    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:51.844925 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:51.844904    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:58:52.818871 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:52.818832    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerStarted","Data":"ddeddd1d724d676e6bb7e926ee366dcd4edb1db5ee1becc73b19b1b11ec3411d"}
Sep 11 03:58:52.819242 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:52.818892    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerStarted","Data":"1f5bbfe7cf7e6032f56be80e7a598a19a8b5d0786e533febffb36298c070a785"}
Sep 11 03:58:52.841066 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:52.841007    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podStartSLOduration=1.840991882 podStartE2EDuration="1.840991882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:58:52.837934642 +0000 UTC m=+1122.480724803" watchObservedRunningTime="2026-09-11 03:58:52.840991882 +0000 UTC m=+1122.483782043"
Sep 11 03:58:53.568076 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:53.568040    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:53.573161 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:53.573135    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:53.825260 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:53.825238    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:53.826173 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:53.826156    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:58:54.003137 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.003111    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:54.118904 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.118800    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvcqz\" (UniqueName: \"kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz\") pod \"53b71449-d721-4323-8006-c52244c5a54b\" (UID: \"53b71449-d721-4323-8006-c52244c5a54b\") "
Sep 11 03:58:54.120911 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.120879    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz" (OuterVolumeSpecName: "kube-api-access-mvcqz") pod "53b71449-d721-4323-8006-c52244c5a54b" (UID: "53b71449-d721-4323-8006-c52244c5a54b"). InnerVolumeSpecName "kube-api-access-mvcqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 03:58:54.220059 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.220030    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvcqz\" (UniqueName: \"kubernetes.io/projected/53b71449-d721-4323-8006-c52244c5a54b-kube-api-access-mvcqz\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 03:58:54.512475 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.512413    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:54.742249 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.742210    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:54.831475 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.831386    2553 generic.go:356] "Generic (PLEG): container finished" podID="53b71449-d721-4323-8006-c52244c5a54b" containerID="ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1" exitCode=137
Sep 11 03:58:54.831475 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.831427    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:54.832004 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.831486    2553 scope.go:117] "RemoveContainer" containerID="ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1"
Sep 11 03:58:54.837395 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.837369    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:54.841686 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.841666    2553 scope.go:117] "RemoveContainer" containerID="ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1"
Sep 11 03:58:54.842042 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:54.842014    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1\": container with ID starting with ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1 not found: ID does not exist" containerID="ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1"
Sep 11 03:58:54.842137 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.842053    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1"} err="failed to get container status \"ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1\": rpc error: code = NotFound desc = could not find container \"ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1\": container with ID starting with ae6e28d909664586aa07610e959ca861b2f7704522b588b786acc55f6f5c95a1 not found: ID does not exist"
Sep 11 03:58:54.843320 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:54.843291    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7701-1aa382\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="53b71449-d721-4323-8006-c52244c5a54b" pod="opendatahub/test-curl-7701-1aa382"
Sep 11 03:58:54.872928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:54.872903    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53b71449-d721-4323-8006-c52244c5a54b" path="/var/lib/kubelet/pods/53b71449-d721-4323-8006-c52244c5a54b/volumes"
Sep 11 03:58:55.229663 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:55.229578    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:55.844964 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:55.844937    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:56.580721 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:56.580681    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.190396 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.190360    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 03:58:57.204551 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.204485    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.206698 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.206673    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 03:58:57.207651 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.207629    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-89230689-serving-cert\""
Sep 11 03:58:57.225257 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.225233    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.310665 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.310632    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.349973 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.349939    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.350142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.350007    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.350142 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.350065    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.350233 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.350141    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blskp\" (UniqueName: \"kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.451306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.451228    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.451306 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.451304    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.451454 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:57.451327    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-89230689-serving-cert: secret "maas-api-e2e-mt-89230689-serving-cert" not found
Sep 11 03:58:57.451454 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.451342    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.451454 ip-10-0-139-129 kubenswrapper[2553]: E0911 03:58:57.451400    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls podName:7e54b557-41ee-4562-a6fd-a8da0a140502 nodeName:}" failed. No retries permitted until 2026-09-11 03:58:57.951381037 +0000 UTC m=+1127.594171179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls") pod "maas-api-e2e-mt-89230689-59d4499c58-pqx5h" (UID: "7e54b557-41ee-4562-a6fd-a8da0a140502") : secret "maas-api-e2e-mt-89230689-serving-cert" not found
Sep 11 03:58:57.451601 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.451481    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blskp\" (UniqueName: \"kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.451919 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.451900    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.453649 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.453631    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.466810 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.466785    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blskp\" (UniqueName: \"kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.589668 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.589638    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.781005 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.780970    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.850525 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.850489    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:58:57.955578 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.955544    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:57.958840 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:57.958794    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") pod \"maas-api-e2e-mt-89230689-59d4499c58-pqx5h\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:58.117443 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:58.117366    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:58.392311 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:58.392279    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 03:58:58.392909 ip-10-0-139-129 kubenswrapper[2553]: W0911 03:58:58.392883    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e54b557_41ee_4562_a6fd_a8da0a140502.slice/crio-883d7d18ac074fc93b5caa9e0231e4e2c1b4a44ede77dbf3b933899ea8d9b61a WatchSource:0}: Error finding container 883d7d18ac074fc93b5caa9e0231e4e2c1b4a44ede77dbf3b933899ea8d9b61a: Status 404 returned error can't find the container with id 883d7d18ac074fc93b5caa9e0231e4e2c1b4a44ede77dbf3b933899ea8d9b61a
Sep 11 03:58:58.860656 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:58.860621    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" event={"ID":"7e54b557-41ee-4562-a6fd-a8da0a140502","Type":"ContainerStarted","Data":"883d7d18ac074fc93b5caa9e0231e4e2c1b4a44ede77dbf3b933899ea8d9b61a"}
Sep 11 03:58:59.868473 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:59.868437    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" event={"ID":"7e54b557-41ee-4562-a6fd-a8da0a140502","Type":"ContainerStarted","Data":"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"}
Sep 11 03:58:59.868864 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:59.868578    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:58:59.887071 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:58:59.887022    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" podStartSLOduration=2.369293082 podStartE2EDuration="2.887007485s" totalImagesPullingTime="517.714403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 03:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 03:58:59.88367062 +0000 UTC m=+1129.526460782" watchObservedRunningTime="2026-09-11 03:58:59.887007485 +0000 UTC m=+1129.529797698"
Sep 11 03:59:05.879671 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:05.879637    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 03:59:14.952895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.952801    2553 generic.go:356] "Generic (PLEG): container finished" podID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerID="ddeddd1d724d676e6bb7e926ee366dcd4edb1db5ee1becc73b19b1b11ec3411d" exitCode=0
Sep 11 03:59:14.952895 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.952874    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerDied","Data":"ddeddd1d724d676e6bb7e926ee366dcd4edb1db5ee1becc73b19b1b11ec3411d"}
Sep 11 03:59:14.953333 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.953314    2553 scope.go:117] "RemoveContainer" containerID="ddeddd1d724d676e6bb7e926ee366dcd4edb1db5ee1becc73b19b1b11ec3411d"
Sep 11 03:59:14.954452 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.954423    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:59:14.954508 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.954495    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:59:14.954547 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:14.954531    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 03:59:15.965749 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:15.965714    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerStarted","Data":"408c9c1a2c0593968744ec096e9829ad28cc4479519626fbbef7ac9192ab22e4"}
Sep 11 03:59:16.568450 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:16.568409    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:59:16.568646 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:16.568621    2553 patch_prober.go:28] interesting pod/e2e-mt-89230689-openshift-default-554676cc96-nf7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 03:59:16.568695 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:16.568672    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 03:59:16.908941 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:16.908852    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:17.568297 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:17.568258    2553 patch_prober.go:28] interesting pod/e2e-mt-89230689-openshift-default-554676cc96-nf7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 03:59:17.568778 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:17.568319    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 03:59:18.568481 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:18.568432    2553 patch_prober.go:28] interesting pod/e2e-mt-89230689-openshift-default-554676cc96-nf7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 03:59:18.569052 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:18.568507    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 03:59:19.164251 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:19.164216    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:19.568721 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:19.568684    2553 patch_prober.go:28] interesting pod/e2e-mt-89230689-openshift-default-554676cc96-nf7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 03:59:19.569112 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:19.568740    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 03:59:19.768939 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:19.768893    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:20.042928 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:20.042895    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:20.568492 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:20.568452    2553 patch_prober.go:28] interesting pod/e2e-mt-89230689-openshift-default-554676cc96-nf7hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Sep 11 03:59:20.568669 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:20.568515    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Sep 11 03:59:20.904587 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:20.904498    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:21.508708 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:21.508672    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:21.572523 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:21.572496    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:59:21.572864 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:21.572841    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:59:21.573970 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:21.573937    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 03:59:22.215048 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:22.215009    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:22.805200 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:22.805165    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:44.122243 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:44.122162    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 03:59:49.203417 ip-10-0-139-129 kubenswrapper[2553]: I0911 03:59:49.203387    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:10.041484 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:10.041455    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:10.507586 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:10.507493    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:10.905870 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:10.905790    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:00:10.912976 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:10.912951    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:00:11.219473 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:11.219377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:11.839514 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:11.839477    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:12.459692 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:12.459658    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:13.225641 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:13.225593    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:13.837858 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:13.837797    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:25.061999 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.061953    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 04:00:25.062642 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.062203    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" podUID="7e54b557-41ee-4562-a6fd-a8da0a140502" containerName="maas-api" containerID="cri-o://61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568" gracePeriod=30
Sep 11 04:00:25.090315 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.090279    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:25.189126 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.189091    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:25.289364 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.289331    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:25.322176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.322117    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 04:00:25.346879 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.346849    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:25.378420 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.378370    2553 generic.go:356] "Generic (PLEG): container finished" podID="7e54b557-41ee-4562-a6fd-a8da0a140502" containerID="61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568" exitCode=0
Sep 11 04:00:25.378628 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.378567    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" event={"ID":"7e54b557-41ee-4562-a6fd-a8da0a140502","Type":"ContainerDied","Data":"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"}
Sep 11 04:00:25.378628 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.378586    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"
Sep 11 04:00:25.378743 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.378626    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h" event={"ID":"7e54b557-41ee-4562-a6fd-a8da0a140502","Type":"ContainerDied","Data":"883d7d18ac074fc93b5caa9e0231e4e2c1b4a44ede77dbf3b933899ea8d9b61a"}
Sep 11 04:00:25.378743 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.378657    2553 scope.go:117] "RemoveContainer" containerID="61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"
Sep 11 04:00:25.387199 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387171    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca\") pod \"7e54b557-41ee-4562-a6fd-a8da0a140502\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") "
Sep 11 04:00:25.387340 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387225    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blskp\" (UniqueName: \"kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp\") pod \"7e54b557-41ee-4562-a6fd-a8da0a140502\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") "
Sep 11 04:00:25.387340 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387326    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") pod \"7e54b557-41ee-4562-a6fd-a8da0a140502\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") "
Sep 11 04:00:25.387458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387362    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs\") pod \"7e54b557-41ee-4562-a6fd-a8da0a140502\" (UID: \"7e54b557-41ee-4562-a6fd-a8da0a140502\") "
Sep 11 04:00:25.387541 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387509    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e54b557-41ee-4562-a6fd-a8da0a140502" (UID: "7e54b557-41ee-4562-a6fd-a8da0a140502"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:00:25.387775 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.387666    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e54b557-41ee-4562-a6fd-a8da0a140502-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:00:25.390379 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.390358    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e54b557-41ee-4562-a6fd-a8da0a140502" (UID: "7e54b557-41ee-4562-a6fd-a8da0a140502"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:00:25.390872 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.390845    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp" (OuterVolumeSpecName: "kube-api-access-blskp") pod "7e54b557-41ee-4562-a6fd-a8da0a140502" (UID: "7e54b557-41ee-4562-a6fd-a8da0a140502"). InnerVolumeSpecName "kube-api-access-blskp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:00:25.391090 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.391066    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e54b557-41ee-4562-a6fd-a8da0a140502" (UID: "7e54b557-41ee-4562-a6fd-a8da0a140502"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:00:25.393996 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.393967    2553 scope.go:117] "RemoveContainer" containerID="61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"
Sep 11 04:00:25.394384 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:00:25.394355    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568\": container with ID starting with 61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568 not found: ID does not exist" containerID="61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"
Sep 11 04:00:25.394477 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.394396    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568"} err="failed to get container status \"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568\": rpc error: code = NotFound desc = could not find container \"61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568\": container with ID starting with 61e320bb508cce628b56fe1d42d6ba61f133bb245ca2ef853e8116b8f855f568 not found: ID does not exist"
Sep 11 04:00:25.489055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.489021    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:00:25.489055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.489049    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e54b557-41ee-4562-a6fd-a8da0a140502-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:00:25.489055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.489059    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blskp\" (UniqueName: \"kubernetes.io/projected/7e54b557-41ee-4562-a6fd-a8da0a140502-kube-api-access-blskp\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:00:25.701601 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.701544    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 04:00:25.705243 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.705218    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89230689-59d4499c58-pqx5h"]
Sep 11 04:00:25.737920 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:25.737892    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:26.359641 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:26.359586    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:26.873540 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:26.873502    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e54b557-41ee-4562-a6fd-a8da0a140502" path="/var/lib/kubelet/pods/7e54b557-41ee-4562-a6fd-a8da0a140502/volumes"
Sep 11 04:00:27.086216 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:27.086186    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:27.918281 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:27.918240    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:28.505054 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:28.505014    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:28.605839 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:28.605794    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:00:33.514200 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:00:33.514169    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:10.211433 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:10.211331    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:12.005847 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:12.005794    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:15.739409 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:15.739365    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:27.052366 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:27.052330    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:28.191437 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:28.191405    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:30.361144 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:30.361115    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:30.435994 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:30.435957    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:30.615531 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:30.615446    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 04:01:30.615694 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:30.615669    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy" containerID="cri-o://408c9c1a2c0593968744ec096e9829ad28cc4479519626fbbef7ac9192ab22e4" gracePeriod=30
Sep 11 04:01:30.642399 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:30.642350    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:31.999523 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:31.999495    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:32.141303 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:32.141267    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:32.803967 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:32.803930    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:32.898538 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:32.898504    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:32.998993 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:32.998960    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:34.133396 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:34.133366    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:34.307958 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:34.307921    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:35.789272 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.789227    2553 generic.go:356] "Generic (PLEG): container finished" podID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerID="408c9c1a2c0593968744ec096e9829ad28cc4479519626fbbef7ac9192ab22e4" exitCode=0
Sep 11 04:01:35.789646 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.789298    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerDied","Data":"408c9c1a2c0593968744ec096e9829ad28cc4479519626fbbef7ac9192ab22e4"}
Sep 11 04:01:35.789646 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.789378    2553 scope.go:117] "RemoveContainer" containerID="ddeddd1d724d676e6bb7e926ee366dcd4edb1db5ee1becc73b19b1b11ec3411d"
Sep 11 04:01:35.861171 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.861145    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 04:01:35.989876 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.989785    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990037 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.989875    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990037 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.989954    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v6hr\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990037 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990003    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990037 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990024    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990258 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990053    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990258 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990089    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990258 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990123    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990258 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990155    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy\") pod \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\" (UID: \"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be\") "
Sep 11 04:01:35.990468 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990317    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:01:35.990528 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990478    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:01:35.990528 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990480    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:01:35.990638 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990611    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:01:35.990696 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990667    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data" (OuterVolumeSpecName: "istio-data") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:01:35.990810 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990732    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:35.990810 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.990761    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:35.992663 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.992636    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token" (OuterVolumeSpecName: "istio-token") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:01:35.992663 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.992649    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:01:35.992663 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.992655    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr" (OuterVolumeSpecName: "kube-api-access-7v6hr") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "kube-api-access-7v6hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:01:35.992849 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:35.992680    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" (UID: "e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:01:36.091344 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091306    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v6hr\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-kube-api-access-7v6hr\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091344 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091339    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091344 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091351    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091360    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091369    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091379    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.091573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.091388    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:01:36.798108 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.798077    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"
Sep 11 04:01:36.798594 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.798122    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw" event={"ID":"e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be","Type":"ContainerDied","Data":"1f5bbfe7cf7e6032f56be80e7a598a19a8b5d0786e533febffb36298c070a785"}
Sep 11 04:01:36.798594 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.798182    2553 scope.go:117] "RemoveContainer" containerID="408c9c1a2c0593968744ec096e9829ad28cc4479519626fbbef7ac9192ab22e4"
Sep 11 04:01:36.823729 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.823702    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 04:01:36.828930 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.828908    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-89230689-openshift-default-554676cc96-nf7hw"]
Sep 11 04:01:36.874680 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:36.874633    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" path="/var/lib/kubelet/pods/e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be/volumes"
Sep 11 04:01:40.161239 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:40.161206    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:40.235281 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:40.235250    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:40.947682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:40.947643    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:46.577624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:46.577582    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:46.687288 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:46.687253    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:46.781213 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:46.781182    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:57.280138 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:57.280098    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:01:57.514573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:01:57.514539    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:12.905281 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:12.905250    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:18.507169 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:18.507137    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:24.425997 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:24.425952    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:24.507025 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:24.506991    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:24.791521 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:24.791486    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:25.098973 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.098892    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:02:25.100027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100001    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:25.100188 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100175    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:25.100273 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100263    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e54b557-41ee-4562-a6fd-a8da0a140502" containerName="maas-api"
Sep 11 04:02:25.100348 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100339    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e54b557-41ee-4562-a6fd-a8da0a140502" containerName="maas-api"
Sep 11 04:02:25.100654 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100638    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:25.100763 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100742    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e54b557-41ee-4562-a6fd-a8da0a140502" containerName="maas-api"
Sep 11 04:02:25.100763 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.100766    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:25.106109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.106088    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.109408 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.109388    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-49518b-openshift-default-dockercfg-r84m9\""
Sep 11 04:02:25.127995 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.127968    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:02:25.133905 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.133877    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:25.227691 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227656    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227691 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227691    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227715    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227790    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227855    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227889    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.227921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227911    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrtl4\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.228086 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.227963    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.228086 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.228013    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329043 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329010    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329183 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329048    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329183 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329083    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329183 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329126    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329183 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329173    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329365 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329221    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329365 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329251    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrtl4\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329365 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329282    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329365 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329313    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329754 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329731    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.329928 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329743    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.330067 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.329843    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.330197 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.330135    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.330291 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.330272    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.331637 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.331616    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.331928 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.331910    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.345467 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.345437    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrtl4\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.349297 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.349238    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token\") pod \"e2e-api-a-49518b-openshift-default-66665585bb-22mjv\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") " pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.390837 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.390801    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:25.416790 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.416766    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:25.700275 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.700199    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:02:25.701946 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:02:25.701916    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11305f89_d740_4322_abcd_0acb5ae4bb5d.slice/crio-4324daf8f7e617d6e6506388b54f9c0e2de2e561d28aee05ca91ab74fb545da5 WatchSource:0}: Error finding container 4324daf8f7e617d6e6506388b54f9c0e2de2e561d28aee05ca91ab74fb545da5: Status 404 returned error can't find the container with id 4324daf8f7e617d6e6506388b54f9c0e2de2e561d28aee05ca91ab74fb545da5
Sep 11 04:02:25.704720 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.704690    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:25.704780 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.704757    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:25.704839 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:25.704783    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:26.079856 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:26.079806    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerStarted","Data":"4699fa1ee53b339a3ec1631c95a879420aae4606a003a654e1a51801b24db530"}
Sep 11 04:02:26.080054 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:26.079885    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerStarted","Data":"4324daf8f7e617d6e6506388b54f9c0e2de2e561d28aee05ca91ab74fb545da5"}
Sep 11 04:02:26.104047 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:26.104002    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podStartSLOduration=1.103987698 podStartE2EDuration="1.103987698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:02:26.100160101 +0000 UTC m=+1335.742950347" watchObservedRunningTime="2026-09-11 04:02:26.103987698 +0000 UTC m=+1335.746777860"
Sep 11 04:02:26.417096 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:26.417009    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:26.422110 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:26.422085    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:27.085880 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:27.085850    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:27.086728 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:27.086707    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:27.917916 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:27.917883    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:28.089561 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:28.089527    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:28.200044 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:28.199969    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:29.239306 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:29.239273    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:30.087246 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:30.087214    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:30.836066 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:30.836032    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:31.814109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.814079    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:02:31.814508 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.814495    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:31.814558 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.814511    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16bf9f4-9622-4e6b-b22d-bdf90a4dd0be" containerName="istio-proxy"
Sep 11 04:02:31.818135 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.818108    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:31.822766 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.822745    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-49518b-serving-cert\""
Sep 11 04:02:31.836922 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.836894    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:02:31.859693 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.859659    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:31.898885 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.898834    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:31.899087 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.898952    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:31.899087 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.899001    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfrdk\" (UniqueName: \"kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:31.899204 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.899137    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:31.972325 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:31.972282    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:32.000559 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.000513    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.000750 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.000570    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfrdk\" (UniqueName: \"kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.000750 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.000621    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.000750 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.000642    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.000750 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:02:32.000747    2553 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-serving-cert: secret "maas-api-e2e-api-a-49518b-serving-cert" not found
Sep 11 04:02:32.001029 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:02:32.000840    2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls podName:d1d373ce-cb26-43f7-8048-31f2ad58d320 nodeName:}" failed. No retries permitted until 2026-09-11 04:02:32.500793326 +0000 UTC m=+1342.143583478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls") pod "maas-api-e2e-api-a-49518b-7485486fdf-46pt4" (UID: "d1d373ce-cb26-43f7-8048-31f2ad58d320") : secret "maas-api-e2e-api-a-49518b-serving-cert" not found
Sep 11 04:02:32.001376 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.001351    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.003023 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.003001    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.010042 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.010018    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfrdk\" (UniqueName: \"kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.162589 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.162501    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:32.415948 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.415833    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:32.505860 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.505794    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.508188 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.508156    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") pod \"maas-api-e2e-api-a-49518b-7485486fdf-46pt4\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:32.529409 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.529374    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:32.730219 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:32.730128    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:33.012652 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:33.012630    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:02:33.014442 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:02:33.014416    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d373ce_cb26_43f7_8048_31f2ad58d320.slice/crio-fbd2d62b1e30b1d706e0aeaa0cdb0c20bb8ee7025757c08dee4b139abf1d9c99 WatchSource:0}: Error finding container fbd2d62b1e30b1d706e0aeaa0cdb0c20bb8ee7025757c08dee4b139abf1d9c99: Status 404 returned error can't find the container with id fbd2d62b1e30b1d706e0aeaa0cdb0c20bb8ee7025757c08dee4b139abf1d9c99
Sep 11 04:02:33.015831 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:33.015800    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:02:33.128868 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:33.128790    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" event={"ID":"d1d373ce-cb26-43f7-8048-31f2ad58d320","Type":"ContainerStarted","Data":"fbd2d62b1e30b1d706e0aeaa0cdb0c20bb8ee7025757c08dee4b139abf1d9c99"}
Sep 11 04:02:34.136015 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:34.135979    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" event={"ID":"d1d373ce-cb26-43f7-8048-31f2ad58d320","Type":"ContainerStarted","Data":"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"}
Sep 11 04:02:34.136421 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:34.136076    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:34.156825 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:34.156759    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" podStartSLOduration=2.705471895 podStartE2EDuration="3.156740653s" totalImagesPullingTime="451.268758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:02:34.155433025 +0000 UTC m=+1343.798223198" watchObservedRunningTime="2026-09-11 04:02:34.156740653 +0000 UTC m=+1343.799530846"
Sep 11 04:02:39.251331 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:39.251291    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:40.154157 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:40.154111    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:02:49.229868 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.229809    2553 generic.go:356] "Generic (PLEG): container finished" podID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerID="4699fa1ee53b339a3ec1631c95a879420aae4606a003a654e1a51801b24db530" exitCode=0
Sep 11 04:02:49.230457 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.229886    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerDied","Data":"4699fa1ee53b339a3ec1631c95a879420aae4606a003a654e1a51801b24db530"}
Sep 11 04:02:49.230457 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.230443    2553 scope.go:117] "RemoveContainer" containerID="4699fa1ee53b339a3ec1631c95a879420aae4606a003a654e1a51801b24db530"
Sep 11 04:02:49.231665 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.231637    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:49.231725 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.231714    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:49.231761 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:49.231746    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:02:50.237542 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:50.237509    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerStarted","Data":"20a4a3a56bd96582eb5dd927b7b934d29b80ac9a4f5ca1bdb40cd4fae058baf8"}
Sep 11 04:02:50.417802 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:50.417761    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:50.418087 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:50.418061    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 04:02:50.418178 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:50.418133    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 04:02:51.417903 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:51.417863    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 04:02:51.418379 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:51.417928    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 04:02:52.418055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:52.418014    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 04:02:52.418444 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:52.418081    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 04:02:53.417918 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:53.417881    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 04:02:53.418095 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:53.417938    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 04:02:54.418158 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:54.418119    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 04:02:54.418546 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:54.418182    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 04:02:54.533182 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:54.533138    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:55.545002 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:55.544969    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:55.545428 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:55.545279    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:55.546039 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:55.546019    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:02:56.596952 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:56.596920    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:57.438587 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:57.438557    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:58.216152 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:58.216112    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:58.574201 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:58.574167    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:58.705800 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:58.705763    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:59.255720 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:59.255685    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:59.646727 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:59.646692    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:02:59.992376 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:02:59.992295    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:01.093740 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:01.093703    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:02.112001 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:02.111959    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:02.527415 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:02.527376    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:02.615785 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:02.615747    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:02.949546 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:02.949455    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:03.803107 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.803071    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:03:03.806445 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.806422    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:03.809291 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.809270    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b61c46-serving-cert\""
Sep 11 04:03:03.819104 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.819073    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:03:03.837292 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.837221    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:03.913279 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.913237    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:03.926251 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.926216    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:03.926416 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.926260    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:03.926416 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.926307    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:03.926416 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:03.926391    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4mm\" (UniqueName: \"kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.012087 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.012043    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:04.027878 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.027852    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4mm\" (UniqueName: \"kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.028020 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.027995    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.028063 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.028024    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.028063 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.028056    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.028635 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.028599    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.030525 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.030503    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.030616 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.030553    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.037889 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.037860    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4mm\" (UniqueName: \"kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm\") pod \"maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.119894 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.119776    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:04.367068 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.367018    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:04.403339 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.403310    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:03:04.404440 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:03:04.404417    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134a113b_547e_42f9_a182_c04aa2a0fb6d.slice/crio-3439d42dc3d05bc9b72461ffff072ab63560aa3bb4fcb8d242d0ef7376092600 WatchSource:0}: Error finding container 3439d42dc3d05bc9b72461ffff072ab63560aa3bb4fcb8d242d0ef7376092600: Status 404 returned error can't find the container with id 3439d42dc3d05bc9b72461ffff072ab63560aa3bb4fcb8d242d0ef7376092600
Sep 11 04:03:04.463588 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.463556    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:04.932201 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:04.932169    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:05.339898 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:05.339863    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" event={"ID":"134a113b-547e-42f9-a182-c04aa2a0fb6d","Type":"ContainerStarted","Data":"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"}
Sep 11 04:03:05.340116 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:05.339926    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" event={"ID":"134a113b-547e-42f9-a182-c04aa2a0fb6d","Type":"ContainerStarted","Data":"3439d42dc3d05bc9b72461ffff072ab63560aa3bb4fcb8d242d0ef7376092600"}
Sep 11 04:03:05.340116 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:05.340064    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:05.364473 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:05.364417    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" podStartSLOduration=1.896022127 podStartE2EDuration="2.364401536s" totalImagesPullingTime="468.379409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:03:05.362088643 +0000 UTC m=+1375.004878805" watchObservedRunningTime="2026-09-11 04:03:05.364401536 +0000 UTC m=+1375.007191749"
Sep 11 04:03:05.916438 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:05.916399    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:06.605982 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.605944    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7704-fd74ac"]
Sep 11 04:03:06.608949 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.608923    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:06.611613 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.611591    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fjqk5\""
Sep 11 04:03:06.616926 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.616896    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-fd74ac"]
Sep 11 04:03:06.627427 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.627403    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:06.757951 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.757912    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztsc6\" (UniqueName: \"kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6\") pod \"test-curl-7704-fd74ac\" (UID: \"470bac7b-2cb9-4442-abae-fb314154b049\") " pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:06.859485 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.859392    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztsc6\" (UniqueName: \"kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6\") pod \"test-curl-7704-fd74ac\" (UID: \"470bac7b-2cb9-4442-abae-fb314154b049\") " pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:06.868759 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.868730    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztsc6\" (UniqueName: \"kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6\") pod \"test-curl-7704-fd74ac\" (UID: \"470bac7b-2cb9-4442-abae-fb314154b049\") " pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:06.920240 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:06.920210    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:07.202072 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:07.202047    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7704-fd74ac"]
Sep 11 04:03:07.203157 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:03:07.203135    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod470bac7b_2cb9_4442_abae_fb314154b049.slice/crio-1bcdcf26b3259d86869f262c6b24e609f666803e8b21e8a1b406d820cccd9669 WatchSource:0}: Error finding container 1bcdcf26b3259d86869f262c6b24e609f666803e8b21e8a1b406d820cccd9669: Status 404 returned error can't find the container with id 1bcdcf26b3259d86869f262c6b24e609f666803e8b21e8a1b406d820cccd9669
Sep 11 04:03:07.354939 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:07.354903    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-fd74ac" event={"ID":"470bac7b-2cb9-4442-abae-fb314154b049","Type":"ContainerStarted","Data":"cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d"}
Sep 11 04:03:07.354939 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:07.354949    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7704-fd74ac" event={"ID":"470bac7b-2cb9-4442-abae-fb314154b049","Type":"ContainerStarted","Data":"1bcdcf26b3259d86869f262c6b24e609f666803e8b21e8a1b406d820cccd9669"}
Sep 11 04:03:07.381101 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:07.381045    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7704-fd74ac" podStartSLOduration=1.3810297679999999 podStartE2EDuration="1.381029768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:03:07.378868003 +0000 UTC m=+1377.021658167" watchObservedRunningTime="2026-09-11 04:03:07.381029768 +0000 UTC m=+1377.023819930"
Sep 11 04:03:07.485138 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:07.485054    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:08.063490 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:08.063456    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7704-fd74ac"]
Sep 11 04:03:08.067619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:08.067583    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7704-fd74ac"]
Sep 11 04:03:08.361209 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:03:08.361181    2553 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7704-fd74ac" container="test-curl-7704-fd74ac"
Sep 11 04:03:08.441736 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:08.441699    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:09.367140 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:09.367095    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7704-fd74ac" podUID="470bac7b-2cb9-4442-abae-fb314154b049" containerName="test-curl-7704-fd74ac" containerID="cri-o://cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d" gracePeriod=2
Sep 11 04:03:11.353103 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.353069    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:03:11.566412 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.566383    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:11.705347 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.705271    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztsc6\" (UniqueName: \"kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6\") pod \"470bac7b-2cb9-4442-abae-fb314154b049\" (UID: \"470bac7b-2cb9-4442-abae-fb314154b049\") "
Sep 11 04:03:11.707301 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.707278    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6" (OuterVolumeSpecName: "kube-api-access-ztsc6") pod "470bac7b-2cb9-4442-abae-fb314154b049" (UID: "470bac7b-2cb9-4442-abae-fb314154b049"). InnerVolumeSpecName "kube-api-access-ztsc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:03:11.737110 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.737080    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:11.806682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:11.806646    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztsc6\" (UniqueName: \"kubernetes.io/projected/470bac7b-2cb9-4442-abae-fb314154b049-kube-api-access-ztsc6\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:12.126143 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.126109    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:12.385883 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.385763    2553 generic.go:356] "Generic (PLEG): container finished" podID="470bac7b-2cb9-4442-abae-fb314154b049" containerID="cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d" exitCode=137
Sep 11 04:03:12.385883 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.385809    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:12.385883 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.385875    2553 scope.go:117] "RemoveContainer" containerID="cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d"
Sep 11 04:03:12.395613 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.395596    2553 scope.go:117] "RemoveContainer" containerID="cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d"
Sep 11 04:03:12.395931 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:03:12.395913    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d\": container with ID starting with cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d not found: ID does not exist" containerID="cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d"
Sep 11 04:03:12.395973 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.395941    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d"} err="failed to get container status \"cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d\": rpc error: code = NotFound desc = could not find container \"cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d\": container with ID starting with cae3d1c483e085b249b7d5caaa08b0562aa4ae72796a6f52c84af6d31bf5d22d not found: ID does not exist"
Sep 11 04:03:12.396849 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:03:12.396798    2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7704-fd74ac\" is forbidden: User \"system:node:ip-10-0-139-129.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-129.ec2.internal' and this object" podUID="470bac7b-2cb9-4442-abae-fb314154b049" pod="opendatahub/test-curl-7704-fd74ac"
Sep 11 04:03:12.713159 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.713073    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:12.872419 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:12.872382    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="470bac7b-2cb9-4442-abae-fb314154b049" path="/var/lib/kubelet/pods/470bac7b-2cb9-4442-abae-fb314154b049/volumes"
Sep 11 04:03:24.837840 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:24.837777    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:26.227110 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:26.227080    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:26.876282 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:26.876248    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:27.420780 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:27.420739    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:27.718957 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:27.718861    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:28.388836 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:28.388777    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:29.109218 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:29.109175    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:29.822055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:29.822023    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:30.622331 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:30.622297    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:31.336536 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:31.336500    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:31.988598 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:31.988534    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:32.840060 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:32.840014    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:33.348709 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:33.348676    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:33.449128 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:33.449090    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:33.556378 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:33.556343    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:34.292888 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:34.292847    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:34.833660 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:34.833628    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:35.097609 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:35.097527    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:35.827787 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:35.827754    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:40.629040 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:40.629002    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:41.265115 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:41.265059    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:46.810566 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:46.810531    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:03:46.811064 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:46.810777    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" podUID="d1d373ce-cb26-43f7-8048-31f2ad58d320" containerName="maas-api" containerID="cri-o://5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4" gracePeriod=30
Sep 11 04:03:46.841938 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:46.841896    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:46.910137 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:46.910094    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:46.974199 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:46.974161    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:47.042294 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.042262    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:47.067418 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.067365    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:03:47.164985 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.164948    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca\") pod \"d1d373ce-cb26-43f7-8048-31f2ad58d320\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") "
Sep 11 04:03:47.165187 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.165022    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") pod \"d1d373ce-cb26-43f7-8048-31f2ad58d320\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") "
Sep 11 04:03:47.165187 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.165179    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfrdk\" (UniqueName: \"kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk\") pod \"d1d373ce-cb26-43f7-8048-31f2ad58d320\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") "
Sep 11 04:03:47.165318 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.165223    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs\") pod \"d1d373ce-cb26-43f7-8048-31f2ad58d320\" (UID: \"d1d373ce-cb26-43f7-8048-31f2ad58d320\") "
Sep 11 04:03:47.165398 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.165356    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d1d373ce-cb26-43f7-8048-31f2ad58d320" (UID: "d1d373ce-cb26-43f7-8048-31f2ad58d320"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:03:47.165606 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.165570    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1d373ce-cb26-43f7-8048-31f2ad58d320-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:47.167443 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.167417    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d1d373ce-cb26-43f7-8048-31f2ad58d320" (UID: "d1d373ce-cb26-43f7-8048-31f2ad58d320"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:03:47.167570 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.167497    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk" (OuterVolumeSpecName: "kube-api-access-jfrdk") pod "d1d373ce-cb26-43f7-8048-31f2ad58d320" (UID: "d1d373ce-cb26-43f7-8048-31f2ad58d320"). InnerVolumeSpecName "kube-api-access-jfrdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:03:47.167709 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.167693    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d1d373ce-cb26-43f7-8048-31f2ad58d320" (UID: "d1d373ce-cb26-43f7-8048-31f2ad58d320"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:03:47.266573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.266537    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:47.266573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.266565    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d1d373ce-cb26-43f7-8048-31f2ad58d320-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:47.266573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.266574    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfrdk\" (UniqueName: \"kubernetes.io/projected/d1d373ce-cb26-43f7-8048-31f2ad58d320-kube-api-access-jfrdk\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:47.412954 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.412866    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:47.528056 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.528026    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:47.603733 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.603692    2553 generic.go:356] "Generic (PLEG): container finished" podID="d1d373ce-cb26-43f7-8048-31f2ad58d320" containerID="5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4" exitCode=0
Sep 11 04:03:47.603970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.603745    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"
Sep 11 04:03:47.603970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.603797    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" event={"ID":"d1d373ce-cb26-43f7-8048-31f2ad58d320","Type":"ContainerDied","Data":"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"}
Sep 11 04:03:47.603970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.603926    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4" event={"ID":"d1d373ce-cb26-43f7-8048-31f2ad58d320","Type":"ContainerDied","Data":"fbd2d62b1e30b1d706e0aeaa0cdb0c20bb8ee7025757c08dee4b139abf1d9c99"}
Sep 11 04:03:47.603970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.603959    2553 scope.go:117] "RemoveContainer" containerID="5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"
Sep 11 04:03:47.614508 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.614490    2553 scope.go:117] "RemoveContainer" containerID="5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"
Sep 11 04:03:47.614792 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:03:47.614774    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4\": container with ID starting with 5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4 not found: ID does not exist" containerID="5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"
Sep 11 04:03:47.614888 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.614801    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4"} err="failed to get container status \"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4\": rpc error: code = NotFound desc = could not find container \"5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4\": container with ID starting with 5da58fff5c662ccfe04d796480e981fdc1a1d3e8c58b2475b226d4e9c530aae4 not found: ID does not exist"
Sep 11 04:03:47.629177 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.629154    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:03:47.634852 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:47.634806    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-49518b-7485486fdf-46pt4"]
Sep 11 04:03:48.191015 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:48.190980    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:48.841551 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:48.841510    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:48.874248 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:48.874208    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d373ce-cb26-43f7-8048-31f2ad58d320" path="/var/lib/kubelet/pods/d1d373ce-cb26-43f7-8048-31f2ad58d320/volumes"
Sep 11 04:03:49.648134 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:49.648101    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:50.267990 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:50.267955    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:51.039824 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:51.039779    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:53.198636 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:53.198601    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:53.298459 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:53.298415    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:53.437851 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:53.437786    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:03:53.438418 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:53.438385    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" containerID="cri-o://20a4a3a56bd96582eb5dd927b7b934d29b80ac9a4f5ca1bdb40cd4fae058baf8" gracePeriod=30
Sep 11 04:03:53.455582 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:53.455500    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:54.514046 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:54.514009    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:55.332089 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:55.332057    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:55.547890 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:55.547850    2553 patch_prober.go:28] interesting pod/e2e-api-a-49518b-openshift-default-66665585bb-22mjv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 04:03:55.548261 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:55.547929    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 04:03:55.971420 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:55.971387    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:56.709025 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:56.708991    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:57.365336 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:57.365304    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:58.001133 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.001096    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:03:58.671540 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.671506    2553 generic.go:356] "Generic (PLEG): container finished" podID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerID="20a4a3a56bd96582eb5dd927b7b934d29b80ac9a4f5ca1bdb40cd4fae058baf8" exitCode=0
Sep 11 04:03:58.671540 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.671532    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerDied","Data":"20a4a3a56bd96582eb5dd927b7b934d29b80ac9a4f5ca1bdb40cd4fae058baf8"}
Sep 11 04:03:58.671796 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.671641    2553 scope.go:117] "RemoveContainer" containerID="4699fa1ee53b339a3ec1631c95a879420aae4606a003a654e1a51801b24db530"
Sep 11 04:03:58.706361 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.706337    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:03:58.778792 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.778750    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.778792 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.778802    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779012 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.778960    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779062 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779033    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779062 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779056    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779163 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779090    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrtl4\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779163 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779110    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779163 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779142    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779323 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779159    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:03:58.779323 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779180    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data\") pod \"11305f89-d740-4322-abcd-0acb5ae4bb5d\" (UID: \"11305f89-d740-4322-abcd-0acb5ae4bb5d\") "
Sep 11 04:03:58.779323 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779194    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:03:58.779488 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779415    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:03:58.779551 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779488    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:03:58.779551 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779506    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.779551 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779527    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11305f89-d740-4322-abcd-0acb5ae4bb5d-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.779551 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779540    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.779704 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.779569    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:03:58.781348 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.781313    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4" (OuterVolumeSpecName: "kube-api-access-qrtl4") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "kube-api-access-qrtl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:03:58.781474 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.781322    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:03:58.781474 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.781357    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:03:58.781591 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.781513    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11305f89-d740-4322-abcd-0acb5ae4bb5d" (UID: "11305f89-d740-4322-abcd-0acb5ae4bb5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880395    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrtl4\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-kube-api-access-qrtl4\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880431    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880447    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880458    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880469    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:58.880624 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:58.880480    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11305f89-d740-4322-abcd-0acb5ae4bb5d-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:03:59.679172 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:59.679146    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"
Sep 11 04:03:59.679649 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:59.679188    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv" event={"ID":"11305f89-d740-4322-abcd-0acb5ae4bb5d","Type":"ContainerDied","Data":"4324daf8f7e617d6e6506388b54f9c0e2de2e561d28aee05ca91ab74fb545da5"}
Sep 11 04:03:59.679649 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:59.679247    2553 scope.go:117] "RemoveContainer" containerID="20a4a3a56bd96582eb5dd927b7b934d29b80ac9a4f5ca1bdb40cd4fae058baf8"
Sep 11 04:03:59.697272 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:59.697244    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:03:59.701092 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:03:59.701071    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-49518b-openshift-default-66665585bb-22mjv"]
Sep 11 04:04:00.872712 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:00.872683    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" path="/var/lib/kubelet/pods/11305f89-d740-4322-abcd-0acb5ae4bb5d/volumes"
Sep 11 04:04:07.702846 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:07.702757    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.213310 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.213270    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:04:09.213775 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.213589    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" podUID="134a113b-547e-42f9-a182-c04aa2a0fb6d" containerName="maas-api" containerID="cri-o://d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189" gracePeriod=30
Sep 11 04:04:09.226716 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.226680    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.328189 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.328157    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.433215 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.433166    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.468938 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.468877    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:04:09.541687 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.541652    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.582430 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.582398    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls\") pod \"134a113b-547e-42f9-a182-c04aa2a0fb6d\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") "
Sep 11 04:04:09.582625 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.582442    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca\") pod \"134a113b-547e-42f9-a182-c04aa2a0fb6d\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") "
Sep 11 04:04:09.582625 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.582463    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs\") pod \"134a113b-547e-42f9-a182-c04aa2a0fb6d\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") "
Sep 11 04:04:09.582625 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.582525    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz4mm\" (UniqueName: \"kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm\") pod \"134a113b-547e-42f9-a182-c04aa2a0fb6d\" (UID: \"134a113b-547e-42f9-a182-c04aa2a0fb6d\") "
Sep 11 04:04:09.583307 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.583276    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "134a113b-547e-42f9-a182-c04aa2a0fb6d" (UID: "134a113b-547e-42f9-a182-c04aa2a0fb6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:04:09.584803 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.584778    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "134a113b-547e-42f9-a182-c04aa2a0fb6d" (UID: "134a113b-547e-42f9-a182-c04aa2a0fb6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:04:09.585147 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.585102    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "134a113b-547e-42f9-a182-c04aa2a0fb6d" (UID: "134a113b-547e-42f9-a182-c04aa2a0fb6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:04:09.585268 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.585143    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm" (OuterVolumeSpecName: "kube-api-access-sz4mm") pod "134a113b-547e-42f9-a182-c04aa2a0fb6d" (UID: "134a113b-547e-42f9-a182-c04aa2a0fb6d"). InnerVolumeSpecName "kube-api-access-sz4mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:04:09.684113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.684073    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/134a113b-547e-42f9-a182-c04aa2a0fb6d-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:09.684113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.684108    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:09.684113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.684118    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz4mm\" (UniqueName: \"kubernetes.io/projected/134a113b-547e-42f9-a182-c04aa2a0fb6d-kube-api-access-sz4mm\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:09.684359 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.684127    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/134a113b-547e-42f9-a182-c04aa2a0fb6d-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:09.736389 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.736297    2553 generic.go:356] "Generic (PLEG): container finished" podID="134a113b-547e-42f9-a182-c04aa2a0fb6d" containerID="d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189" exitCode=0
Sep 11 04:04:09.736389 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.736355    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"
Sep 11 04:04:09.736389 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.736378    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" event={"ID":"134a113b-547e-42f9-a182-c04aa2a0fb6d","Type":"ContainerDied","Data":"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"}
Sep 11 04:04:09.736670 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.736449    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b" event={"ID":"134a113b-547e-42f9-a182-c04aa2a0fb6d","Type":"ContainerDied","Data":"3439d42dc3d05bc9b72461ffff072ab63560aa3bb4fcb8d242d0ef7376092600"}
Sep 11 04:04:09.736670 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.736472    2553 scope.go:117] "RemoveContainer" containerID="d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"
Sep 11 04:04:09.747725 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.747698    2553 scope.go:117] "RemoveContainer" containerID="d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"
Sep 11 04:04:09.748076 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:04:09.748048    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189\": container with ID starting with d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189 not found: ID does not exist" containerID="d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"
Sep 11 04:04:09.748166 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.748088    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189"} err="failed to get container status \"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189\": rpc error: code = NotFound desc = could not find container \"d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189\": container with ID starting with d643f99bd9a5d4c4bc1f59bb7e046662af2b89b01964ebc1c68319ee98f09189 not found: ID does not exist"
Sep 11 04:04:09.760059 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.760031    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:04:09.765047 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.765020    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b61c46-dc6b946f9-f6c9b"]
Sep 11 04:04:09.816272 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.816238    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:09.916116 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:09.916084    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:10.617507 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:10.617473    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:10.872297 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:10.872225    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="134a113b-547e-42f9-a182-c04aa2a0fb6d" path="/var/lib/kubelet/pods/134a113b-547e-42f9-a182-c04aa2a0fb6d/volumes"
Sep 11 04:04:11.451034 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:11.450993    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:12.146676 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:12.146645    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:12.788804 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:12.788771    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:15.575498 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:15.575463    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:15.650588 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:15.650558    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:15.840481 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:15.840377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:17.142270 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:17.142236    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:17.879383 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:17.879351    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:18.689462 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:18.689428    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:19.327288 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:19.327250    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:19.953516 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:19.953476    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:21.174739 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:21.174703    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:21.241562 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:21.241531    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:26.628560 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:26.628527    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:26.728847 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:26.728798    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:26.810275 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:26.810229    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:26.865462 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:26.865416    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:27.240118 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:27.240078    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:27.341169 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:27.341132    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:27.856926 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:27.856890    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:28.631131 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:28.631092    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:29.351487 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:29.351454    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:32.994066 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:32.994030    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:33.052810 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:33.052778    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:33.239380 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:33.239342    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 04:04:33.239608 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:33.239584    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy" containerID="cri-o://e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf" gracePeriod=30
Sep 11 04:04:33.260098 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:33.260029    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:34.431897 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:34.431857    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:35.114733 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:35.114700    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:35.833389 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:35.833351    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:36.521036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:36.520998    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:38.495810 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.495787    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 04:04:38.551619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551595    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551744 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551634    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551744 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551674    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551744 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551701    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551744 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551740    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rcbr\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551980 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551767    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551980 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551802    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551980 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551896    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:04:38.551980 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551917    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.551980 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551965    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo\") pod \"141039e4-ef92-4d6b-b587-15b436e03b1d\" (UID: \"141039e4-ef92-4d6b-b587-15b436e03b1d\") "
Sep 11 04:04:38.552173 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.551994    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:04:38.552173 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552023    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:04:38.552173 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552120    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:04:38.552173 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552153    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:04:38.552381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552254    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.552381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552271    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.552381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552286    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.552381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552299    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/141039e4-ef92-4d6b-b587-15b436e03b1d-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.552381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.552309    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.554093 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.554058    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:04:38.554214 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.554094    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:04:38.554385 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.554366    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:04:38.554430 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.554386    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr" (OuterVolumeSpecName: "kube-api-access-6rcbr") pod "141039e4-ef92-4d6b-b587-15b436e03b1d" (UID: "141039e4-ef92-4d6b-b587-15b436e03b1d"). InnerVolumeSpecName "kube-api-access-6rcbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:04:38.652918 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.652893    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rcbr\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-kube-api-access-6rcbr\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.652918 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.652914    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.653056 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.652924    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.653056 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.652933    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/141039e4-ef92-4d6b-b587-15b436e03b1d-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:38.900369 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.900340    2553 generic.go:356] "Generic (PLEG): container finished" podID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerID="e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf" exitCode=0
Sep 11 04:04:38.900486 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.900387    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"
Sep 11 04:04:38.900486 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.900423    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerDied","Data":"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"}
Sep 11 04:04:38.900486 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.900477    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz" event={"ID":"141039e4-ef92-4d6b-b587-15b436e03b1d","Type":"ContainerDied","Data":"2c05fd663d370a2318cca427227f165c9cbf003f877e711cfeb17b00759fe0d8"}
Sep 11 04:04:38.900612 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.900514    2553 scope.go:117] "RemoveContainer" containerID="e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"
Sep 11 04:04:38.909634 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.909617    2553 scope.go:117] "RemoveContainer" containerID="4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"
Sep 11 04:04:38.917837 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.917795    2553 scope.go:117] "RemoveContainer" containerID="e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"
Sep 11 04:04:38.918112 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:04:38.918089    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf\": container with ID starting with e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf not found: ID does not exist" containerID="e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"
Sep 11 04:04:38.918165 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.918122    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf"} err="failed to get container status \"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf\": rpc error: code = NotFound desc = could not find container \"e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf\": container with ID starting with e887c811e8b6b7388d476953998f8d795a3e426fb3c62fe7acf71c7bc19ac8cf not found: ID does not exist"
Sep 11 04:04:38.918165 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.918138    2553 scope.go:117] "RemoveContainer" containerID="4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"
Sep 11 04:04:38.918244 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.918190    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 04:04:38.918406 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:04:38.918389    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e\": container with ID starting with 4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e not found: ID does not exist" containerID="4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"
Sep 11 04:04:38.918441 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.918411    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e"} err="failed to get container status \"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e\": rpc error: code = NotFound desc = could not find container \"4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e\": container with ID starting with 4a5a98159ee2c0b23f52a556c3831511ff5a7b62cf8c9bb59529b33210b8fe9e not found: ID does not exist"
Sep 11 04:04:38.922112 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:38.922083    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-20ae9a-openshift-default-7655ccfc9d-gnqnz"]
Sep 11 04:04:40.873071 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:40.873036    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" path="/var/lib/kubelet/pods/141039e4-ef92-4d6b-b587-15b436e03b1d/volumes"
Sep 11 04:04:43.998752 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:43.998711    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 04:04:43.999223 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:43.998966    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" podUID="11d28784-cc4f-4e03-8b2b-b6350e44261f" containerName="maas-api" containerID="cri-o://33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b" gracePeriod=30
Sep 11 04:04:44.024573 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.024535    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.093691 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.093656    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.159970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.159938    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.218027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.217992    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.252248 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.252196    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 04:04:44.294395 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.294337    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca\") pod \"11d28784-cc4f-4e03-8b2b-b6350e44261f\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") "
Sep 11 04:04:44.296836 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.296740    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs\") pod \"11d28784-cc4f-4e03-8b2b-b6350e44261f\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") "
Sep 11 04:04:44.297236 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.297099    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz54f\" (UniqueName: \"kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f\") pod \"11d28784-cc4f-4e03-8b2b-b6350e44261f\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") "
Sep 11 04:04:44.297552 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.297452    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") pod \"11d28784-cc4f-4e03-8b2b-b6350e44261f\" (UID: \"11d28784-cc4f-4e03-8b2b-b6350e44261f\") "
Sep 11 04:04:44.299214 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.299192    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11d28784-cc4f-4e03-8b2b-b6350e44261f" (UID: "11d28784-cc4f-4e03-8b2b-b6350e44261f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:04:44.301938 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.301915    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11d28784-cc4f-4e03-8b2b-b6350e44261f" (UID: "11d28784-cc4f-4e03-8b2b-b6350e44261f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:04:44.304874 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.304749    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11d28784-cc4f-4e03-8b2b-b6350e44261f" (UID: "11d28784-cc4f-4e03-8b2b-b6350e44261f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:04:44.304987 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.304870    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f" (OuterVolumeSpecName: "kube-api-access-jz54f") pod "11d28784-cc4f-4e03-8b2b-b6350e44261f" (UID: "11d28784-cc4f-4e03-8b2b-b6350e44261f"). InnerVolumeSpecName "kube-api-access-jz54f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:04:44.398965 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.398932    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:44.398965 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.398959    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d28784-cc4f-4e03-8b2b-b6350e44261f-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:44.398965 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.398969    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11d28784-cc4f-4e03-8b2b-b6350e44261f-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:44.399190 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.398977    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz54f\" (UniqueName: \"kubernetes.io/projected/11d28784-cc4f-4e03-8b2b-b6350e44261f-kube-api-access-jz54f\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:04:44.580344 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.580304    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.683184 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.683150    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:44.933907 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.933868    2553 generic.go:356] "Generic (PLEG): container finished" podID="11d28784-cc4f-4e03-8b2b-b6350e44261f" containerID="33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b" exitCode=0
Sep 11 04:04:44.934055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.933915    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"
Sep 11 04:04:44.934055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.933960    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" event={"ID":"11d28784-cc4f-4e03-8b2b-b6350e44261f","Type":"ContainerDied","Data":"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"}
Sep 11 04:04:44.934055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.934016    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m" event={"ID":"11d28784-cc4f-4e03-8b2b-b6350e44261f","Type":"ContainerDied","Data":"3e75403a689133d251f59bcbc5b9200666f496b8ac0fccbb26990d8a49cf1abd"}
Sep 11 04:04:44.934055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.934040    2553 scope.go:117] "RemoveContainer" containerID="33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"
Sep 11 04:04:44.942968 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.942949    2553 scope.go:117] "RemoveContainer" containerID="33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"
Sep 11 04:04:44.943252 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:04:44.943235    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b\": container with ID starting with 33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b not found: ID does not exist" containerID="33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"
Sep 11 04:04:44.943311 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.943260    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b"} err="failed to get container status \"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b\": rpc error: code = NotFound desc = could not find container \"33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b\": container with ID starting with 33a435ab095963a1d59242fb30f7dfcdadd0ec7a3b7249f5e65056d1655b295b not found: ID does not exist"
Sep 11 04:04:44.954743 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.954703    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 04:04:44.958526 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:44.958504    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-7dc787-5c787dc86-zf24m"]
Sep 11 04:04:45.421939 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:45.421906    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:46.146886 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:46.146844    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:46.872962 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:46.872926    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d28784-cc4f-4e03-8b2b-b6350e44261f" path="/var/lib/kubelet/pods/11d28784-cc4f-4e03-8b2b-b6350e44261f/volumes"
Sep 11 04:04:49.937692 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:49.937653    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:50.542042 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:50.542008    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:50.641683 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:50.641650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:50.784767 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:50.784735    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:51.403716 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:51.403682    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:52.303264 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:52.303230    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:52.767040 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:52.767005    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:53.048126 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048053    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:04:53.048510 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048489    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048511    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048519    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048524    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048539    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048545    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.048556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048554    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048559    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048568    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d373ce-cb26-43f7-8048-31f2ad58d320" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048573    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d373ce-cb26-43f7-8048-31f2ad58d320" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048579    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="134a113b-547e-42f9-a182-c04aa2a0fb6d" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048584    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="134a113b-547e-42f9-a182-c04aa2a0fb6d" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048591    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d28784-cc4f-4e03-8b2b-b6350e44261f" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048595    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d28784-cc4f-4e03-8b2b-b6350e44261f" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048605    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="470bac7b-2cb9-4442-abae-fb314154b049" containerName="test-curl-7704-fd74ac"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048611    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="470bac7b-2cb9-4442-abae-fb314154b049" containerName="test-curl-7704-fd74ac"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048668    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="134a113b-547e-42f9-a182-c04aa2a0fb6d" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048679    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="470bac7b-2cb9-4442-abae-fb314154b049" containerName="test-curl-7704-fd74ac"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048687    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048692    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d28784-cc4f-4e03-8b2b-b6350e44261f" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048700    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048706    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d373ce-cb26-43f7-8048-31f2ad58d320" containerName="maas-api"
Sep 11 04:04:53.048781 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.048711    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="11305f89-d740-4322-abcd-0acb5ae4bb5d" containerName="istio-proxy"
Sep 11 04:04:53.052866 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.052846    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.056272 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.056250    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4cb558-openshift-default-dockercfg-49khm\""
Sep 11 04:04:53.063732 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.063706    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:04:53.081763 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.081736    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:53.088592 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088566    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088693 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088601    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088693 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088684    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088780 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088724    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088780 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088755    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088901 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088779    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088901 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088797    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088901 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088827    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.088901 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.088852    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189671 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189636    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189671 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189675    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189709    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189731    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189751    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189768    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189785    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189840    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.189953 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.189935    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.190333 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.190087    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.190400 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.190374    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.190458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.190420    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.190519 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.190507    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.190634 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.190614    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.192604 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.192583    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.192755 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.192736    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.197417 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.197390    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.197770 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.197752    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d\") pod \"e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") " pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.365219 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.365138    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:53.600300 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.600266    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:53.641143 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.641113    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:04:53.641989 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:04:53.641960    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5d1532_9b45_4d5e_945c_732774f2bda3.slice/crio-39525100fc0a1cc9face83bdb807d465ac0dda81400c0e57c9813f369d8de914 WatchSource:0}: Error finding container 39525100fc0a1cc9face83bdb807d465ac0dda81400c0e57c9813f369d8de914: Status 404 returned error can't find the container with id 39525100fc0a1cc9face83bdb807d465ac0dda81400c0e57c9813f369d8de914
Sep 11 04:04:53.644315 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.644279    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:04:53.644418 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.644345    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:04:53.644418 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.644376    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:04:53.988942 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.988840    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerStarted","Data":"19d6420087bc8b5169b9cb25595e9c8c79e915aa40c90c28dc1dff7df949010e"}
Sep 11 04:04:53.988942 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:53.988905    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerStarted","Data":"39525100fc0a1cc9face83bdb807d465ac0dda81400c0e57c9813f369d8de914"}
Sep 11 04:04:54.010189 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:54.010141    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podStartSLOduration=1.010126313 podStartE2EDuration="1.010126313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:04:54.009013109 +0000 UTC m=+1483.651803284" watchObservedRunningTime="2026-09-11 04:04:54.010126313 +0000 UTC m=+1483.652916474"
Sep 11 04:04:54.365586 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:54.365551    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:54.370491 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:54.370467    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:54.994657 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:54.994625    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:54.995597 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:54.995579    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:04:55.726131 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:55.726077    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:55.911144 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:55.911108    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:56.003658 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:56.003571    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:56.500550 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:56.500514    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:57.333156 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:57.333122    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:57.397891 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:57.397827    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:57.704464 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:57.704371    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:57.804832 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:57.804780    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:04:57.883637 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:04:57.883593    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:10.948256 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:10.948227    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:05:10.955960 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:10.955938    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:05:11.037031 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:11.036994    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 04:05:11.037198 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:11.037087    2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 04:05:13.106537 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.106493    2553 generic.go:356] "Generic (PLEG): container finished" podID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerID="19d6420087bc8b5169b9cb25595e9c8c79e915aa40c90c28dc1dff7df949010e" exitCode=0
Sep 11 04:05:13.106963 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.106573    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerDied","Data":"19d6420087bc8b5169b9cb25595e9c8c79e915aa40c90c28dc1dff7df949010e"}
Sep 11 04:05:13.107009 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.106979    2553 scope.go:117] "RemoveContainer" containerID="19d6420087bc8b5169b9cb25595e9c8c79e915aa40c90c28dc1dff7df949010e"
Sep 11 04:05:13.108034 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.108007    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:05:13.108092 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.108077    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:05:13.108129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:13.108110    2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 04:05:14.114455 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:14.114421    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerStarted","Data":"990edfb158d3e4d66f7efc84b9f53be99b89f91b36a0e20a83a2875172628ac6"}
Sep 11 04:05:14.366349 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:14.366255    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:05:14.366509 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:14.366432    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 04:05:14.366509 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:14.366477    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 04:05:15.365930 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:15.365892    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 04:05:15.366320 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:15.365977    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 04:05:16.366241 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:16.366202    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 04:05:16.366716 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:16.366267    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 04:05:17.366123 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:17.366088    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 04:05:17.366307 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:17.366160    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 04:05:18.366353 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:18.366315    2553 patch_prober.go:28] interesting pod/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 11 04:05:18.366735 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:18.366389    2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 11 04:05:19.369545 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:19.369510    2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:05:19.369954 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:19.369792    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:05:19.370430 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:19.370412    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:05:19.987140 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:19.987103    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:20.088481 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:20.088444    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:20.188528 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:20.188476    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:20.734598 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:20.734562    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:20.937683 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:20.937649    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:21.474392 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:21.474353    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:21.636183 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:21.636152    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:21.870834 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:21.870789    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:21.956595 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:21.956560    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:22.105553 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:22.105522    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:24.091054 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:24.091019    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:24.795943 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:24.795905    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:24.896192 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:24.896163    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:25.757975 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:25.757941    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:26.617292 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:26.617256    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.232508 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.232479    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:05:27.232982 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.232968    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="141039e4-ef92-4d6b-b587-15b436e03b1d" containerName="istio-proxy"
Sep 11 04:05:27.236312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.236297    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.239077 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.239054    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-0f85b8-serving-cert\""
Sep 11 04:05:27.248045 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.248021    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:05:27.266188 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.266161    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.327770 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.327734    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.327971 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.327800    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.327971 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.327866    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.327971 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.327930    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txtc8\" (UniqueName: \"kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.366725 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.366686    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.428456 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.428418    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.428658 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.428520    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txtc8\" (UniqueName: \"kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.428658 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.428607    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.428806 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.428658    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.429032 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.429003    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.429373 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.429353    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.431162 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.431140    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.431253 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.431181    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.438436 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.438412    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txtc8\" (UniqueName: \"kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8\") pod \"maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.549899 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.549862    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:27.714636 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.714604    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.813928 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.813840    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:27.823726 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:27.823696    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:05:27.824856 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:05:27.824799    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode152a51c_fed3_41f5_9dac_6899318ee278.slice/crio-a175d70b2cb535d89bddfce29acce2189375af7cc94c847968bac44bf39c7019 WatchSource:0}: Error finding container a175d70b2cb535d89bddfce29acce2189375af7cc94c847968bac44bf39c7019: Status 404 returned error can't find the container with id a175d70b2cb535d89bddfce29acce2189375af7cc94c847968bac44bf39c7019
Sep 11 04:05:28.196661 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:28.196629    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" event={"ID":"e152a51c-fed3-41f5-9dac-6899318ee278","Type":"ContainerStarted","Data":"a175d70b2cb535d89bddfce29acce2189375af7cc94c847968bac44bf39c7019"}
Sep 11 04:05:29.203963 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:29.203926    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" event={"ID":"e152a51c-fed3-41f5-9dac-6899318ee278","Type":"ContainerStarted","Data":"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"}
Sep 11 04:05:29.204441 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:29.204004    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:29.221771 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:29.221716    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" podStartSLOduration=1.888346407 podStartE2EDuration="2.221703585s" totalImagesPullingTime="333.357178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:05:29.219889152 +0000 UTC m=+1518.862679313" watchObservedRunningTime="2026-09-11 04:05:29.221703585 +0000 UTC m=+1518.864493746"
Sep 11 04:05:35.217604 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:35.217526    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:05:49.055725 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:49.055685    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:49.718074 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:49.718039    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:49.817422 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:49.817385    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:50.348745 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:50.348716    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:51.131062 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:51.131023    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:05:51.768530 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:05:51.768489    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:03.921559 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:03.921524    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:36.426332 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:36.426290    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:38.017835 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:38.017781    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:38.824592 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:38.824559    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:39.436622 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:39.436586    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:39.594225 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:39.594184    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:39.685416 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:39.685377    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:40.421437 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:40.421404    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:41.131457 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:41.131410    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:41.789510 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:41.789466    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:42.511592 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.511558    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:06:42.516070 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.516043    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.518475 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.518455    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-ebb9b5-serving-cert\""
Sep 11 04:06:42.525945 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.525778    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:06:42.541095 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.541060    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:42.551556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.551529    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.551682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.551560    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk9n9\" (UniqueName: \"kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.551682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.551598    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.551682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.551641    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.643670 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.643633    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:42.652226 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.652197    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.652427 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.652243    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk9n9\" (UniqueName: \"kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.652427 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.652305    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.652427 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.652375    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.653065 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.653047    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.654786 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.654760    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.654910 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.654807    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.660309 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.660287    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk9n9\" (UniqueName: \"kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9\") pod \"maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.830270 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.830183    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:42.999727 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:42.999690    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:43.107751 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:43.107602    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:06:43.107751 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:43.107629    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:43.108123 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:06:43.108100    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc7b560_a2ff_4b0c_858d_b5de04761eee.slice/crio-04d11e60205db90816c920ef255122da36df80040e6d556c6a2a4176b903ff3b WatchSource:0}: Error finding container 04d11e60205db90816c920ef255122da36df80040e6d556c6a2a4176b903ff3b: Status 404 returned error can't find the container with id 04d11e60205db90816c920ef255122da36df80040e6d556c6a2a4176b903ff3b
Sep 11 04:06:43.640378 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:43.640341    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" event={"ID":"abc7b560-a2ff-4b0c-858d-b5de04761eee","Type":"ContainerStarted","Data":"04d11e60205db90816c920ef255122da36df80040e6d556c6a2a4176b903ff3b"}
Sep 11 04:06:44.651540 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:44.651506    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" event={"ID":"abc7b560-a2ff-4b0c-858d-b5de04761eee","Type":"ContainerStarted","Data":"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"}
Sep 11 04:06:44.652003 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:44.651644    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:06:44.674437 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:44.674379    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" podStartSLOduration=1.857686433 podStartE2EDuration="2.674362632s" totalImagesPullingTime="816.676199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:06:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:06:44.672371285 +0000 UTC m=+1594.315161474" watchObservedRunningTime="2026-09-11 04:06:44.674362632 +0000 UTC m=+1594.317152794"
Sep 11 04:06:49.120788 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:49.120746    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:49.173534 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:49.173502    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:06:50.663145 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:06:50.663114    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:07:05.285157 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:05.285082    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:06.007737 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:06.007703    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:06.711547 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:06.711483    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:07.406752 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:07.406718    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:08.124971 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:08.124942    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:08.763259 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:08.763228    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:09.712171 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:09.712136    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:11.133653 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:11.133613    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:11.927350 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:11.927309    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:12.615472 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:12.615431    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:13.233644 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:13.233608    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:14.115560 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:14.115527    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:14.790001 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:14.789959    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.280455 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.280408    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:07:21.281255 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.281202    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" podUID="abc7b560-a2ff-4b0c-858d-b5de04761eee" containerName="maas-api" containerID="cri-o://d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063" gracePeriod=30
Sep 11 04:07:21.316197 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.316147    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.416153 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.416123    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.495506 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.495460    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.532198 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.532144    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:07:21.564522 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.564483    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.634843 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.634791    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls\") pod \"abc7b560-a2ff-4b0c-858d-b5de04761eee\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") "
Sep 11 04:07:21.635014 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.634857    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca\") pod \"abc7b560-a2ff-4b0c-858d-b5de04761eee\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") "
Sep 11 04:07:21.635014 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.634929    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs\") pod \"abc7b560-a2ff-4b0c-858d-b5de04761eee\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") "
Sep 11 04:07:21.635014 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.634985    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk9n9\" (UniqueName: \"kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9\") pod \"abc7b560-a2ff-4b0c-858d-b5de04761eee\" (UID: \"abc7b560-a2ff-4b0c-858d-b5de04761eee\") "
Sep 11 04:07:21.635237 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.635213    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "abc7b560-a2ff-4b0c-858d-b5de04761eee" (UID: "abc7b560-a2ff-4b0c-858d-b5de04761eee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:07:21.635335 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.635313    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/abc7b560-a2ff-4b0c-858d-b5de04761eee-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:07:21.637224 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.636981    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "abc7b560-a2ff-4b0c-858d-b5de04761eee" (UID: "abc7b560-a2ff-4b0c-858d-b5de04761eee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:07:21.637224 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.637049    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "abc7b560-a2ff-4b0c-858d-b5de04761eee" (UID: "abc7b560-a2ff-4b0c-858d-b5de04761eee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:07:21.637224 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.637075    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9" (OuterVolumeSpecName: "kube-api-access-fk9n9") pod "abc7b560-a2ff-4b0c-858d-b5de04761eee" (UID: "abc7b560-a2ff-4b0c-858d-b5de04761eee"). InnerVolumeSpecName "kube-api-access-fk9n9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:07:21.736024 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.735993    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:07:21.736024 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.736020    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/abc7b560-a2ff-4b0c-858d-b5de04761eee-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:07:21.736024 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.736031    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk9n9\" (UniqueName: \"kubernetes.io/projected/abc7b560-a2ff-4b0c-858d-b5de04761eee-kube-api-access-fk9n9\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:07:21.818737 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.818647    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.877241 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.877198    2553 generic.go:356] "Generic (PLEG): container finished" podID="abc7b560-a2ff-4b0c-858d-b5de04761eee" containerID="d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063" exitCode=0
Sep 11 04:07:21.877458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.877357    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" event={"ID":"abc7b560-a2ff-4b0c-858d-b5de04761eee","Type":"ContainerDied","Data":"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"}
Sep 11 04:07:21.877458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.877399    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"
Sep 11 04:07:21.877458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.877426    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns" event={"ID":"abc7b560-a2ff-4b0c-858d-b5de04761eee","Type":"ContainerDied","Data":"04d11e60205db90816c920ef255122da36df80040e6d556c6a2a4176b903ff3b"}
Sep 11 04:07:21.877458 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.877457    2553 scope.go:117] "RemoveContainer" containerID="d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"
Sep 11 04:07:21.889794 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.889776    2553 scope.go:117] "RemoveContainer" containerID="d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"
Sep 11 04:07:21.890083 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:07:21.890063    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063\": container with ID starting with d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063 not found: ID does not exist" containerID="d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"
Sep 11 04:07:21.890145 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.890097    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063"} err="failed to get container status \"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063\": rpc error: code = NotFound desc = could not find container \"d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063\": container with ID starting with d878983ccbac80ad9852afad94179a74963813f0982fe58b88e8ef545e172063 not found: ID does not exist"
Sep 11 04:07:21.902270 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.902248    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:07:21.905275 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.905242    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:21.908143 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:21.908124    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ebb9b5-6658c8b497-q48ns"]
Sep 11 04:07:22.568695 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:22.568665    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:22.878890 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:22.873459    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abc7b560-a2ff-4b0c-858d-b5de04761eee" path="/var/lib/kubelet/pods/abc7b560-a2ff-4b0c-858d-b5de04761eee/volumes"
Sep 11 04:07:23.243552 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:23.243469    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:24.070144 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:24.070104    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:24.714439 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:24.714381    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:27.722146 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:27.722110    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:27.821840 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:27.821786    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:27.993085 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:27.993002    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:32.131970 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:32.131943    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:32.236576 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:32.236536    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:32.337701 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:32.337668    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:32.922346 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:32.922311    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:33.534454 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:33.534422    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:34.196163 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:34.196102    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:35.028485 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:35.028452    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:35.650481 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:35.650425    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:40.134648 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:40.134609    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:42.024108 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:42.024042    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:47.581520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:47.581480    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:47.651113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:47.651076    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:47.752471 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:47.752438    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:47.812510 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:47.812473    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:48.144745 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:48.144706    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:48.244439 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:48.244409    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:48.881381 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:48.881347    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:49.715158 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:49.715122    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:50.327380 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:50.327349    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:53.176369 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:53.176331    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:07:58.288569 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:07:58.288520    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:29.489139 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:29.489106    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:30.319490 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:30.319453    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:34.145164 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:34.145082    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:35.864284 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:35.864234    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:35.965198 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:35.965162    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:36.130569 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:36.130482    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:08:36.130795 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:36.130751    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy" containerID="cri-o://990edfb158d3e4d66f7efc84b9f53be99b89f91b36a0e20a83a2875172628ac6" gracePeriod=30
Sep 11 04:08:36.155392 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:36.155358    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:36.813399 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:36.813340    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:37.519384 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:37.519335    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:38.330358 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:38.330328    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:38.959940 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:38.959903    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:39.607874 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:39.607836    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:41.342078 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.342043    2553 generic.go:356] "Generic (PLEG): container finished" podID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerID="990edfb158d3e4d66f7efc84b9f53be99b89f91b36a0e20a83a2875172628ac6" exitCode=0
Sep 11 04:08:41.342520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.342082    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerDied","Data":"990edfb158d3e4d66f7efc84b9f53be99b89f91b36a0e20a83a2875172628ac6"}
Sep 11 04:08:41.342520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.342144    2553 scope.go:117] "RemoveContainer" containerID="19d6420087bc8b5169b9cb25595e9c8c79e915aa40c90c28dc1dff7df949010e"
Sep 11 04:08:41.393175 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.393115    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:08:41.480836 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.480788    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.480889    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.480915    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.480934    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.480961    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481020    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481058    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481072    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481086    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481213    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token\") pod \"1f5d1532-9b45-4d5e-945c-732774f2bda3\" (UID: \"1f5d1532-9b45-4d5e-945c-732774f2bda3\") "
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481242    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:08:41.481312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481253    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481405    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481425    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481557    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481569    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481579    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481592    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.481629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.481606    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5d1532-9b45-4d5e-945c-732774f2bda3-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.483244 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.483209    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:08:41.483364 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.483292    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:08:41.483522 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.483498    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:08:41.483629 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.483533    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d" (OuterVolumeSpecName: "kube-api-access-qqt9d") pod "1f5d1532-9b45-4d5e-945c-732774f2bda3" (UID: "1f5d1532-9b45-4d5e-945c-732774f2bda3"). InnerVolumeSpecName "kube-api-access-qqt9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:08:41.582936 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.582895    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.582936 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.582929    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.582936 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.582938    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5d1532-9b45-4d5e-945c-732774f2bda3-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:41.582936 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:41.582947    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqt9d\" (UniqueName: \"kubernetes.io/projected/1f5d1532-9b45-4d5e-945c-732774f2bda3-kube-api-access-qqt9d\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:42.348667 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.348633    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"
Sep 11 04:08:42.349151 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.348662    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v" event={"ID":"1f5d1532-9b45-4d5e-945c-732774f2bda3","Type":"ContainerDied","Data":"39525100fc0a1cc9face83bdb807d465ac0dda81400c0e57c9813f369d8de914"}
Sep 11 04:08:42.349151 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.348754    2553 scope.go:117] "RemoveContainer" containerID="990edfb158d3e4d66f7efc84b9f53be99b89f91b36a0e20a83a2875172628ac6"
Sep 11 04:08:42.372427 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.372401    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:08:42.376999 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.376975    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4cb558-openshift-default-5d8d687dd8-84f6v"]
Sep 11 04:08:42.873092 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:42.873057    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" path="/var/lib/kubelet/pods/1f5d1532-9b45-4d5e-945c-732774f2bda3/volumes"
Sep 11 04:08:47.086715 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.086675    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:08:47.087213 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.086911    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" podUID="e152a51c-fed3-41f5-9dac-6899318ee278" containerName="maas-api" containerID="cri-o://92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d" gracePeriod=30
Sep 11 04:08:47.114850 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.114797    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.221450 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.221410    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.328699 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.328664    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.334665 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.334640    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:08:47.379768 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.379660    2553 generic.go:356] "Generic (PLEG): container finished" podID="e152a51c-fed3-41f5-9dac-6899318ee278" containerID="92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d" exitCode=0
Sep 11 04:08:47.379768 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.379725    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" event={"ID":"e152a51c-fed3-41f5-9dac-6899318ee278","Type":"ContainerDied","Data":"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"}
Sep 11 04:08:47.379768 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.379750    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"
Sep 11 04:08:47.380085 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.379798    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf" event={"ID":"e152a51c-fed3-41f5-9dac-6899318ee278","Type":"ContainerDied","Data":"a175d70b2cb535d89bddfce29acce2189375af7cc94c847968bac44bf39c7019"}
Sep 11 04:08:47.380085 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.379855    2553 scope.go:117] "RemoveContainer" containerID="92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"
Sep 11 04:08:47.389985 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.389965    2553 scope.go:117] "RemoveContainer" containerID="92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"
Sep 11 04:08:47.390258 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:08:47.390241    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d\": container with ID starting with 92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d not found: ID does not exist" containerID="92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"
Sep 11 04:08:47.390308 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.390266    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d"} err="failed to get container status \"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d\": rpc error: code = NotFound desc = could not find container \"92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d\": container with ID starting with 92c818638934a560684965bf287e83efec5b2fb54c746912cffb1e4acf361a1d not found: ID does not exist"
Sep 11 04:08:47.431153 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.431124    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.438858 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.438834    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls\") pod \"e152a51c-fed3-41f5-9dac-6899318ee278\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") "
Sep 11 04:08:47.439002 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.438877    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs\") pod \"e152a51c-fed3-41f5-9dac-6899318ee278\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") "
Sep 11 04:08:47.439002 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.438908    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txtc8\" (UniqueName: \"kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8\") pod \"e152a51c-fed3-41f5-9dac-6899318ee278\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") "
Sep 11 04:08:47.439131 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.439037    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca\") pod \"e152a51c-fed3-41f5-9dac-6899318ee278\" (UID: \"e152a51c-fed3-41f5-9dac-6899318ee278\") "
Sep 11 04:08:47.439435 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.439408    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e152a51c-fed3-41f5-9dac-6899318ee278" (UID: "e152a51c-fed3-41f5-9dac-6899318ee278"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:08:47.441076 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.441052    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8" (OuterVolumeSpecName: "kube-api-access-txtc8") pod "e152a51c-fed3-41f5-9dac-6899318ee278" (UID: "e152a51c-fed3-41f5-9dac-6899318ee278"). InnerVolumeSpecName "kube-api-access-txtc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:08:47.441400 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.441384    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e152a51c-fed3-41f5-9dac-6899318ee278" (UID: "e152a51c-fed3-41f5-9dac-6899318ee278"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:08:47.441478 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.441414    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e152a51c-fed3-41f5-9dac-6899318ee278" (UID: "e152a51c-fed3-41f5-9dac-6899318ee278"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:08:47.518114 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.518078    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.542425 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.541042    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e152a51c-fed3-41f5-9dac-6899318ee278-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:47.542425 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.541087    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:47.542425 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.541102    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e152a51c-fed3-41f5-9dac-6899318ee278-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:47.542425 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.541121    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txtc8\" (UniqueName: \"kubernetes.io/projected/e152a51c-fed3-41f5-9dac-6899318ee278-kube-api-access-txtc8\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:08:47.704146 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.704112    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:08:47.709869 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.709845    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-0f85b8-789bf7d847-52zpf"]
Sep 11 04:08:47.820423 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.820389    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:47.920499 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:47.920465    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:48.670978 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:48.670943    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:48.872585 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:48.872541    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e152a51c-fed3-41f5-9dac-6899318ee278" path="/var/lib/kubelet/pods/e152a51c-fed3-41f5-9dac-6899318ee278/volumes"
Sep 11 04:08:49.545169 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:49.545124    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:49.684239 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:49.684207    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:53.678937 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:53.678903    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:53.819061 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:53.819032    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:54.099672 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:54.099641    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:55.116810 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:55.116773    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:55.829850 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:55.829796    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:08:56.530182 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:08:56.530119    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:04.901669 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:04.901634    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 04:09:04.902187 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:04.901937    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" podUID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" containerName="maas-api" containerID="cri-o://d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283" gracePeriod=30
Sep 11 04:09:04.927683 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:04.927645    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:04.999894 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:04.999861    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:05.062938 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.062904    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:05.130879 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.130841    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:05.156004 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.155940    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 04:09:05.313299 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.313271    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs\") pod \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") "
Sep 11 04:09:05.313493 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.313345    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca\") pod \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") "
Sep 11 04:09:05.313493 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.313378    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kzmx\" (UniqueName: \"kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx\") pod \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") "
Sep 11 04:09:05.313493 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.313410    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") pod \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\" (UID: \"291cbbeb-83ae-4b59-84bc-569f9a4cc98b\") "
Sep 11 04:09:05.313828 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.313787    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "291cbbeb-83ae-4b59-84bc-569f9a4cc98b" (UID: "291cbbeb-83ae-4b59-84bc-569f9a4cc98b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:09:05.315650 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.315627    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "291cbbeb-83ae-4b59-84bc-569f9a4cc98b" (UID: "291cbbeb-83ae-4b59-84bc-569f9a4cc98b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:09:05.315755 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.315662    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "291cbbeb-83ae-4b59-84bc-569f9a4cc98b" (UID: "291cbbeb-83ae-4b59-84bc-569f9a4cc98b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:09:05.315855 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.315800    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx" (OuterVolumeSpecName: "kube-api-access-2kzmx") pod "291cbbeb-83ae-4b59-84bc-569f9a4cc98b" (UID: "291cbbeb-83ae-4b59-84bc-569f9a4cc98b"). InnerVolumeSpecName "kube-api-access-2kzmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:09:05.415129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.415041    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:05.415129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.415071    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:05.415129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.415084    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kzmx\" (UniqueName: \"kubernetes.io/projected/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-kube-api-access-2kzmx\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:05.415129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.415096    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/291cbbeb-83ae-4b59-84bc-569f9a4cc98b-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:05.484444 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.484409    2553 generic.go:356] "Generic (PLEG): container finished" podID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" containerID="d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283" exitCode=0
Sep 11 04:09:05.484619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.484451    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" event={"ID":"291cbbeb-83ae-4b59-84bc-569f9a4cc98b","Type":"ContainerDied","Data":"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"}
Sep 11 04:09:05.484619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.484495    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt" event={"ID":"291cbbeb-83ae-4b59-84bc-569f9a4cc98b","Type":"ContainerDied","Data":"3387f24e541082a65979e58c7f0eae33bc940b0de261a40d438c6c21c23595ae"}
Sep 11 04:09:05.484619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.484518    2553 scope.go:117] "RemoveContainer" containerID="d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"
Sep 11 04:09:05.484619 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.484454    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"
Sep 11 04:09:05.499261 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.499239    2553 scope.go:117] "RemoveContainer" containerID="d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"
Sep 11 04:09:05.499566 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:09:05.499544    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283\": container with ID starting with d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283 not found: ID does not exist" containerID="d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"
Sep 11 04:09:05.499617 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.499574    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283"} err="failed to get container status \"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283\": rpc error: code = NotFound desc = could not find container \"d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283\": container with ID starting with d9e98405c7e6520829375830b24bb0eca011c4347159ed558d72a87399d54283 not found: ID does not exist"
Sep 11 04:09:05.510682 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.510658    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 04:09:05.513844 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.513806    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-b4d3cf-77ddd9b7dd-v26xt"]
Sep 11 04:09:05.517375 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.517354    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:05.588418 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:05.588383    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:06.212283 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:06.212245    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:06.874225 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:06.874193    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" path="/var/lib/kubelet/pods/291cbbeb-83ae-4b59-84bc-569f9a4cc98b/volumes"
Sep 11 04:09:11.776346 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:11.776311    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:11.840036 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:11.839984    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:12.060578 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:12.060484    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 04:09:12.060748 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:12.060706    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy" containerID="cri-o://f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308" gracePeriod=30
Sep 11 04:09:12.077939 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:12.077901    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:13.195594 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:13.195559    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:13.924748 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:13.924715    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:17.322071 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.322046    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 04:09:17.324117 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324100    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324173 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324138    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324211 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324172    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324254 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324234    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324266    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324312 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324293    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324415 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324319    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtt8c\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324415 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324402    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324436    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo\") pod \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\" (UID: \"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2\") "
Sep 11 04:09:17.324520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324486    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:09:17.324520 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324504    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:09:17.324687 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324608    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:09:17.324687 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324620    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:09:17.324910 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324889    2553 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istiod-ca-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.325027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324912    2553 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-credential-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.325027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324913    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:09:17.325027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324925    2553 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-socket\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.325027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.324944    2553 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-data\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.326506 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.326459    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 04:09:17.326671 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.326651    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:09:17.326947 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.326924    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:09:17.327004 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.326931    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c" (OuterVolumeSpecName: "kube-api-access-jtt8c") pod "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" (UID: "3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2"). InnerVolumeSpecName "kube-api-access-jtt8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:09:17.425944 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.425914    2553 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-envoy\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.425944 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.425940    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtt8c\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-kube-api-access-jtt8c\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.425944 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.425950    2553 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-workload-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.426162 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.425959    2553 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-podinfo\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.426162 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.425967    2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2-istio-token\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:17.551014 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.550978    2553 generic.go:356] "Generic (PLEG): container finished" podID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerID="f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308" exitCode=0
Sep 11 04:09:17.551176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.551024    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"
Sep 11 04:09:17.551176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.551060    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerDied","Data":"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"}
Sep 11 04:09:17.551176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.551137    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm" event={"ID":"3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2","Type":"ContainerDied","Data":"2ee14a5f418fe2a68187b712ba8ecf0ebc750c7f8f139bc3d3749b4029fe7314"}
Sep 11 04:09:17.551176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.551160    2553 scope.go:117] "RemoveContainer" containerID="f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"
Sep 11 04:09:17.560508 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.560494    2553 scope.go:117] "RemoveContainer" containerID="eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"
Sep 11 04:09:17.570181 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.570166    2553 scope.go:117] "RemoveContainer" containerID="f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"
Sep 11 04:09:17.570426 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:09:17.570409    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308\": container with ID starting with f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308 not found: ID does not exist" containerID="f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"
Sep 11 04:09:17.570477 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.570434    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308"} err="failed to get container status \"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308\": rpc error: code = NotFound desc = could not find container \"f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308\": container with ID starting with f14c6a47906a841beefce1727341c28897e59e2b3120045ab26371669b593308 not found: ID does not exist"
Sep 11 04:09:17.570477 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.570450    2553 scope.go:117] "RemoveContainer" containerID="eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"
Sep 11 04:09:17.570687 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:09:17.570668    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3\": container with ID starting with eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3 not found: ID does not exist" containerID="eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"
Sep 11 04:09:17.570748 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.570697    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3"} err="failed to get container status \"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3\": rpc error: code = NotFound desc = could not find container \"eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3\": container with ID starting with eb959961d88828df2ca06215b5d1562f46629c6ad57fc664a445d2d9b3582bc3 not found: ID does not exist"
Sep 11 04:09:17.576744 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.576687    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 04:09:17.583805 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:17.583783    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b4d3cf-openshift-default-5474ff79fb-vcdlm"]
Sep 11 04:09:18.872734 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:18.872704    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" path="/var/lib/kubelet/pods/3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2/volumes"
Sep 11 04:09:22.993927 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:22.993880    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:23.093003 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:23.092953    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:23.196787 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:23.196751    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:23.297230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:23.297195    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:23.602357 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:23.602272    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:23.715402 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:23.715371    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:29.660273 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:29.660239    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:29.756647 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:29.756615    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:29.954371 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:29.954289    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:09:32.653497 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.653451    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 04:09:32.654075 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.653766    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" podUID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" containerName="manager" containerID="cri-o://0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81" gracePeriod=30
Sep 11 04:09:32.898826 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.898786    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 04:09:32.965963 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.965877    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") pod \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") "
Sep 11 04:09:32.966124 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.965966    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4cdj\" (UniqueName: \"kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj\") pod \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") "
Sep 11 04:09:32.966124 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.966015    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs\") pod \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\" (UID: \"40acae36-a37d-46b6-a6ec-80ce96ce5f8a\") "
Sep 11 04:09:32.968012 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.967982    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj" (OuterVolumeSpecName: "kube-api-access-g4cdj") pod "40acae36-a37d-46b6-a6ec-80ce96ce5f8a" (UID: "40acae36-a37d-46b6-a6ec-80ce96ce5f8a"). InnerVolumeSpecName "kube-api-access-g4cdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:09:32.968124 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.968037    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "40acae36-a37d-46b6-a6ec-80ce96ce5f8a" (UID: "40acae36-a37d-46b6-a6ec-80ce96ce5f8a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:09:32.968124 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:32.968054    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "40acae36-a37d-46b6-a6ec-80ce96ce5f8a" (UID: "40acae36-a37d-46b6-a6ec-80ce96ce5f8a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:09:33.067669 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.067636    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4cdj\" (UniqueName: \"kubernetes.io/projected/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-kube-api-access-g4cdj\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:33.067669 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.067666    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:33.067884 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.067676    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/40acae36-a37d-46b6-a6ec-80ce96ce5f8a-webhook-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:09:33.636479 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.636441    2553 generic.go:356] "Generic (PLEG): container finished" podID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" containerID="0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81" exitCode=0
Sep 11 04:09:33.636657 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.636488    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-n9bt7"
Sep 11 04:09:33.636657 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.636526    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" event={"ID":"40acae36-a37d-46b6-a6ec-80ce96ce5f8a","Type":"ContainerDied","Data":"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"}
Sep 11 04:09:33.636657 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.636596    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-n9bt7" event={"ID":"40acae36-a37d-46b6-a6ec-80ce96ce5f8a","Type":"ContainerDied","Data":"1e8bed702d3b253b41cd0f6bf1af8f8592279f8ac4737842adc34da8fd74975c"}
Sep 11 04:09:33.636657 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.636625    2553 scope.go:117] "RemoveContainer" containerID="0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"
Sep 11 04:09:33.646531 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.646515    2553 scope.go:117] "RemoveContainer" containerID="0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"
Sep 11 04:09:33.646776 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:09:33.646756    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81\": container with ID starting with 0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81 not found: ID does not exist" containerID="0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"
Sep 11 04:09:33.646920 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.646785    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81"} err="failed to get container status \"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81\": rpc error: code = NotFound desc = could not find container \"0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81\": container with ID starting with 0a100d53b17a7ca69bdc159fbd13aeeb4e77dd3378ba4da5ddd7dbced7fa6c81 not found: ID does not exist"
Sep 11 04:09:33.658751 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.658721    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 04:09:33.662400 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:33.662379    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-n9bt7"]
Sep 11 04:09:34.873626 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:34.873592    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" path="/var/lib/kubelet/pods/40acae36-a37d-46b6-a6ec-80ce96ce5f8a/volumes"
Sep 11 04:09:57.314076 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:09:57.314045    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:10:10.986968 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:10.986941    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:10:10.997249 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:10.997225    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:10:34.507029 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.506991    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507519    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507538    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507559    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507567    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507584    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abc7b560-a2ff-4b0c-858d-b5de04761eee" containerName="maas-api"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507595    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="abc7b560-a2ff-4b0c-858d-b5de04761eee" containerName="maas-api"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507605    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e152a51c-fed3-41f5-9dac-6899318ee278" containerName="maas-api"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507614    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e152a51c-fed3-41f5-9dac-6899318ee278" containerName="maas-api"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507625    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507634    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507654    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" containerName="manager"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507662    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" containerName="manager"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507678    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" containerName="maas-api"
Sep 11 04:10:34.507730 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507685    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" containerName="maas-api"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507756    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="291cbbeb-83ae-4b59-84bc-569f9a4cc98b" containerName="maas-api"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507767    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="40acae36-a37d-46b6-a6ec-80ce96ce5f8a" containerName="manager"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507772    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507778    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507783    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="abc7b560-a2ff-4b0c-858d-b5de04761eee" containerName="maas-api"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507788    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507794    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3cb689-b5ae-4dfe-8c5a-cad81d5c15d2" containerName="istio-proxy"
Sep 11 04:10:34.508230 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.507798    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e152a51c-fed3-41f5-9dac-6899318ee278" containerName="maas-api"
Sep 11 04:10:34.510703 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.510685    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.514994 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.514971    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 04:10:34.515113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.515013    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 04:10:34.515113 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.515032    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w5z8x\""
Sep 11 04:10:34.515302 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.515289    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 04:10:34.519662 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.519639    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:34.610602 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.610572    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.610776 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.610619    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.610776 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.610639    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbbdm\" (UniqueName: \"kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.711712 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.711662    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.711941 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.711733    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.711941 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.711760    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbbdm\" (UniqueName: \"kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.714223 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.714204    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.714359 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.714337    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.722013 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.721987    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbbdm\" (UniqueName: \"kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm\") pod \"maas-controller-66fc969cd5-brlpx\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") " pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:34.822149 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:34.822115    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:35.101299 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.101276    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:35.103377 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:10:35.103348    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef5cadd_4304_4377_9b82_20fd85e06489.slice/crio-46fd2fddaa0d68fb0efd173e16cbd9755dca536d47b5f31e0e5e4c9f9444523c WatchSource:0}: Error finding container 46fd2fddaa0d68fb0efd173e16cbd9755dca536d47b5f31e0e5e4c9f9444523c: Status 404 returned error can't find the container with id 46fd2fddaa0d68fb0efd173e16cbd9755dca536d47b5f31e0e5e4c9f9444523c
Sep 11 04:10:35.104702 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.104683    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:10:35.979557 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.979466    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-brlpx" event={"ID":"0ef5cadd-4304-4377-9b82-20fd85e06489","Type":"ContainerStarted","Data":"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"}
Sep 11 04:10:35.979557 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.979529    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-brlpx" event={"ID":"0ef5cadd-4304-4377-9b82-20fd85e06489","Type":"ContainerStarted","Data":"46fd2fddaa0d68fb0efd173e16cbd9755dca536d47b5f31e0e5e4c9f9444523c"}
Sep 11 04:10:35.980066 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.979698    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:35.998131 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:35.998090    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66fc969cd5-brlpx" podStartSLOduration=1.504673076 podStartE2EDuration="1.998074442s" totalImagesPullingTime="493.401366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:10:35.995282551 +0000 UTC m=+1825.638072713" watchObservedRunningTime="2026-09-11 04:10:35.998074442 +0000 UTC m=+1825.640864605"
Sep 11 04:10:46.988023 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:46.987985    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:56.251993 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.251954    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:10:56.354101 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.354042    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:56.354341 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.354312    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-66fc969cd5-brlpx" podUID="0ef5cadd-4304-4377-9b82-20fd85e06489" containerName="manager" containerID="cri-o://99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630" gracePeriod=30
Sep 11 04:10:56.602568 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.602544    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:56.704227 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.704196    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs\") pod \"0ef5cadd-4304-4377-9b82-20fd85e06489\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") "
Sep 11 04:10:56.704396 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.704239    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert\") pod \"0ef5cadd-4304-4377-9b82-20fd85e06489\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") "
Sep 11 04:10:56.704396 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.704281    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbbdm\" (UniqueName: \"kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm\") pod \"0ef5cadd-4304-4377-9b82-20fd85e06489\" (UID: \"0ef5cadd-4304-4377-9b82-20fd85e06489\") "
Sep 11 04:10:56.706351 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.706318    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ef5cadd-4304-4377-9b82-20fd85e06489" (UID: "0ef5cadd-4304-4377-9b82-20fd85e06489"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:10:56.706463 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.706355    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0ef5cadd-4304-4377-9b82-20fd85e06489" (UID: "0ef5cadd-4304-4377-9b82-20fd85e06489"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:10:56.706463 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.706374    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm" (OuterVolumeSpecName: "kube-api-access-zbbdm") pod "0ef5cadd-4304-4377-9b82-20fd85e06489" (UID: "0ef5cadd-4304-4377-9b82-20fd85e06489"). InnerVolumeSpecName "kube-api-access-zbbdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:10:56.805193 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.805103    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:10:56.805193 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.805135    2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ef5cadd-4304-4377-9b82-20fd85e06489-webhook-cert\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:10:56.805193 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:56.805147    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbbdm\" (UniqueName: \"kubernetes.io/projected/0ef5cadd-4304-4377-9b82-20fd85e06489-kube-api-access-zbbdm\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:10:57.102609 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.102513    2553 generic.go:356] "Generic (PLEG): container finished" podID="0ef5cadd-4304-4377-9b82-20fd85e06489" containerID="99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630" exitCode=0
Sep 11 04:10:57.102609 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.102553    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-brlpx"
Sep 11 04:10:57.102609 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.102588    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-brlpx" event={"ID":"0ef5cadd-4304-4377-9b82-20fd85e06489","Type":"ContainerDied","Data":"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"}
Sep 11 04:10:57.102880 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.102643    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-brlpx" event={"ID":"0ef5cadd-4304-4377-9b82-20fd85e06489","Type":"ContainerDied","Data":"46fd2fddaa0d68fb0efd173e16cbd9755dca536d47b5f31e0e5e4c9f9444523c"}
Sep 11 04:10:57.102880 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.102673    2553 scope.go:117] "RemoveContainer" containerID="99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"
Sep 11 04:10:57.111662 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.111624    2553 scope.go:117] "RemoveContainer" containerID="99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"
Sep 11 04:10:57.111949 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:10:57.111927    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630\": container with ID starting with 99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630 not found: ID does not exist" containerID="99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"
Sep 11 04:10:57.112028 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.111956    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630"} err="failed to get container status \"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630\": rpc error: code = NotFound desc = could not find container \"99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630\": container with ID starting with 99c023ae8fef4846ffce6c814a9c88f5c891bf8dc730be7eaf08ca25e285a630 not found: ID does not exist"
Sep 11 04:10:57.122908 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.122874    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:57.125100 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.125082    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-brlpx"]
Sep 11 04:10:57.810865 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.810834    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:10:57.926090 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926056    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-66fc969cd5-pkqcp"]
Sep 11 04:10:57.926429 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926416    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ef5cadd-4304-4377-9b82-20fd85e06489" containerName="manager"
Sep 11 04:10:57.926491 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926430    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ef5cadd-4304-4377-9b82-20fd85e06489" containerName="manager"
Sep 11 04:10:57.926491 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926440    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:57.926491 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926445    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5d1532-9b45-4d5e-945c-732774f2bda3" containerName="istio-proxy"
Sep 11 04:10:57.926641 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.926504    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ef5cadd-4304-4377-9b82-20fd85e06489" containerName="manager"
Sep 11 04:10:57.931526 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.931504    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:57.935616 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.935593    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 04:10:57.935741 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.935593    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 04:10:57.935741 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.935662    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 04:10:57.935741 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.935689    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w5z8x\""
Sep 11 04:10:57.940213 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:57.940189    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-pkqcp"]
Sep 11 04:10:58.016504 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.016470    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-metrics-certs\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.016694 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.016558    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-webhook-cert\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.016694 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.016623    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmnkj\" (UniqueName: \"kubernetes.io/projected/54247ecb-72d2-4423-8cd4-b947785e2355-kube-api-access-vmnkj\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.117716 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.117616    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-webhook-cert\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.117716 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.117705    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmnkj\" (UniqueName: \"kubernetes.io/projected/54247ecb-72d2-4423-8cd4-b947785e2355-kube-api-access-vmnkj\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.118027 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.117754    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-metrics-certs\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.120147 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.120111    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-webhook-cert\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.120147 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.120144    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/54247ecb-72d2-4423-8cd4-b947785e2355-metrics-certs\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.125975 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.125951    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmnkj\" (UniqueName: \"kubernetes.io/projected/54247ecb-72d2-4423-8cd4-b947785e2355-kube-api-access-vmnkj\") pod \"maas-controller-66fc969cd5-pkqcp\" (UID: \"54247ecb-72d2-4423-8cd4-b947785e2355\") " pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.245683 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.245644    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:58.529596 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.529569    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-66fc969cd5-pkqcp"]
Sep 11 04:10:58.530944 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:10:58.530919    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54247ecb_72d2_4423_8cd4_b947785e2355.slice/crio-9c7c8cfc2f4eee0499bea16062c8d091b81c80b7ac122b89ba30e042ec85bf83 WatchSource:0}: Error finding container 9c7c8cfc2f4eee0499bea16062c8d091b81c80b7ac122b89ba30e042ec85bf83: Status 404 returned error can't find the container with id 9c7c8cfc2f4eee0499bea16062c8d091b81c80b7ac122b89ba30e042ec85bf83
Sep 11 04:10:58.872892 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:58.872798    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ef5cadd-4304-4377-9b82-20fd85e06489" path="/var/lib/kubelet/pods/0ef5cadd-4304-4377-9b82-20fd85e06489/volumes"
Sep 11 04:10:59.115852 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:59.115792    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-pkqcp" event={"ID":"54247ecb-72d2-4423-8cd4-b947785e2355","Type":"ContainerStarted","Data":"40cba545e6b51d3d2ad821df43338cf80e857d7cc6c041eeeb0c835df7fa2e35"}
Sep 11 04:10:59.116047 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:59.115874    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-66fc969cd5-pkqcp" event={"ID":"54247ecb-72d2-4423-8cd4-b947785e2355","Type":"ContainerStarted","Data":"9c7c8cfc2f4eee0499bea16062c8d091b81c80b7ac122b89ba30e042ec85bf83"}
Sep 11 04:10:59.116047 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:59.115983    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:10:59.140890 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:10:59.137263    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-66fc969cd5-pkqcp" podStartSLOduration=1.769061325 podStartE2EDuration="2.137241981s" totalImagesPullingTime="368.180656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:10:59.132267186 +0000 UTC m=+1848.775057348" watchObservedRunningTime="2026-09-11 04:10:59.137241981 +0000 UTC m=+1848.780032143"
Sep 11 04:11:10.123137 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:11:10.123103    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-66fc969cd5-pkqcp"
Sep 11 04:12:24.494559 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:12:24.494523    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:13:35.062206 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:13:35.062169    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:13:35.125923 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:13:35.125896    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:13:57.824692 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:13:57.824650    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:14:08.738882 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:14:08.738831    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:14:26.487439 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:14:26.487406    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:14:47.540271 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:14:47.540192    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:14:47.641876 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:14:47.641836    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:14:51.320554 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:14:51.320510    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:15:11.027757 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:11.027727    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:15:11.036580 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:11.036561    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:15:13.187728 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:13.187697    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:15:13.976493 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:13.976457    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:15:45.300478 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.300445    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-778c967957-r6g9d"]
Sep 11 04:15:45.303775 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.303758    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.317085 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.317057    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-778c967957-r6g9d"]
Sep 11 04:15:45.400044 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.400003    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9xm\" (UniqueName: \"kubernetes.io/projected/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-kube-api-access-vb9xm\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.400044 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.400045    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-maas-api-tls\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.400240 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.400114    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-openshift-service-ca\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.400240 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.400153    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-metrics-certs\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.501397 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.501360    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-openshift-service-ca\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.501569 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.501424    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-metrics-certs\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.501569 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.501490    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9xm\" (UniqueName: \"kubernetes.io/projected/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-kube-api-access-vb9xm\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.501569 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.501516    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-maas-api-tls\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.502053 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.502024    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-openshift-service-ca\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.503797 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.503777    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-maas-api-tls\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.503887 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.503870    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-metrics-certs\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.512055 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.512037    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9xm\" (UniqueName: \"kubernetes.io/projected/dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5-kube-api-access-vb9xm\") pod \"maas-api-778c967957-r6g9d\" (UID: \"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5\") " pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.613921 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.613847    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:45.887350 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.887323    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-778c967957-r6g9d"]
Sep 11 04:15:45.888463 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:15:45.888438    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2f2e89_fba1_4798_a8e2_02a6dd7ce3d5.slice/crio-5232ae9b27ad3cc210554f6a60bf3074d809c7e9d79226bdb57f00ab364b03e1 WatchSource:0}: Error finding container 5232ae9b27ad3cc210554f6a60bf3074d809c7e9d79226bdb57f00ab364b03e1: Status 404 returned error can't find the container with id 5232ae9b27ad3cc210554f6a60bf3074d809c7e9d79226bdb57f00ab364b03e1
Sep 11 04:15:45.890218 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:45.890186    2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:15:46.685257 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:46.685221    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d" event={"ID":"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5","Type":"ContainerStarted","Data":"a5653ea6f4f5be5ba19159d7e42421e985435c82c0669a6aeb9dd94f45d35ad3"}
Sep 11 04:15:46.685622 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:46.685278    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d" event={"ID":"dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5","Type":"ContainerStarted","Data":"5232ae9b27ad3cc210554f6a60bf3074d809c7e9d79226bdb57f00ab364b03e1"}
Sep 11 04:15:46.685622 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:46.685357    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:46.710006 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:46.709946    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d" podStartSLOduration=1.400096193 podStartE2EDuration="1.709931754s" totalImagesPullingTime="309.835561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:15:46.707102679 +0000 UTC m=+2136.349892854" watchObservedRunningTime="2026-09-11 04:15:46.709931754 +0000 UTC m=+2136.352721916"
Sep 11 04:15:52.695952 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:52.695922    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-778c967957-r6g9d"
Sep 11 04:15:52.792444 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:52.792404    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 04:15:52.792661 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:52.792620    2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" podUID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" containerName="maas-api" containerID="cri-o://056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8" gracePeriod=30
Sep 11 04:15:54.035946 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.035921    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 04:15:54.080870 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.080836    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca\") pod \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") "
Sep 11 04:15:54.081008 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.080894    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs\") pod \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") "
Sep 11 04:15:54.081008 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.080948    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls\") pod \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") "
Sep 11 04:15:54.081138 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.081027    2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5dhn\" (UniqueName: \"kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn\") pod \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\" (UID: \"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd\") "
Sep 11 04:15:54.081431 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.081270    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" (UID: "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:15:54.083138 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.083112    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" (UID: "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:15:54.083238 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.083210    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn" (OuterVolumeSpecName: "kube-api-access-m5dhn") pod "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" (UID: "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd"). InnerVolumeSpecName "kube-api-access-m5dhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:15:54.083280 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.083257    2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" (UID: "06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:15:54.182109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.182028    2553 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-maas-api-tls\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:15:54.182109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.182056    2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5dhn\" (UniqueName: \"kubernetes.io/projected/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-kube-api-access-m5dhn\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:15:54.182109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.182066    2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-openshift-service-ca\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:15:54.182109 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.182075    2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd-metrics-certs\") on node \"ip-10-0-139-129.ec2.internal\" DevicePath \"\""
Sep 11 04:15:54.736452 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.736416    2553 generic.go:356] "Generic (PLEG): container finished" podID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" containerID="056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8" exitCode=0
Sep 11 04:15:54.736626 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.736462    2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb"
Sep 11 04:15:54.736626 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.736497    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" event={"ID":"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd","Type":"ContainerDied","Data":"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"}
Sep 11 04:15:54.736626 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.736562    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577f88484f-svdhb" event={"ID":"06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd","Type":"ContainerDied","Data":"b57251823c2477706727018c9976275e24e4a61c21a6f3c64c7be533484c26cf"}
Sep 11 04:15:54.736626 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.736585    2553 scope.go:117] "RemoveContainer" containerID="056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"
Sep 11 04:15:54.746414 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.746395    2553 scope.go:117] "RemoveContainer" containerID="056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"
Sep 11 04:15:54.746674 ip-10-0-139-129 kubenswrapper[2553]: E0911 04:15:54.746656    2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8\": container with ID starting with 056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8 not found: ID does not exist" containerID="056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"
Sep 11 04:15:54.746725 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.746681    2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8"} err="failed to get container status \"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8\": rpc error: code = NotFound desc = could not find container \"056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8\": container with ID starting with 056d880946cdbb4554f7b90044528746c836bda14c813c7154c979b4799d7aa8 not found: ID does not exist"
Sep 11 04:15:54.766389 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.766362    2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 04:15:54.770149 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.770117    2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-577f88484f-svdhb"]
Sep 11 04:15:54.872955 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:15:54.872920    2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" path="/var/lib/kubelet/pods/06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd/volumes"
Sep 11 04:16:07.220565 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:07.220472    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:16:14.194947 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:14.194900    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hhtzd_e0aacd83-19e1-4965-a283-c82b75a7f1c3/manager/0.log"
Sep 11 04:16:14.323541 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:14.323512    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b9pcn_a41e9d5b-4228-4516-9117-a9282bd84e5c/manager/0.log"
Sep 11 04:16:14.441176 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:14.441134    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66fc969cd5-pkqcp_54247ecb-72d2-4423-8cd4-b947785e2355/manager/0.log"
Sep 11 04:16:14.557936 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:14.557906    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6jngc_01c0b92a-72df-401d-be77-d5656b6accaa/server/0.log"
Sep 11 04:16:14.682541 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:14.682509    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9fscw_e1290a39-9e24-4ebf-8d61-70e9bbad7590/manager/0.log"
Sep 11 04:16:15.065557 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:15.065525    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q_afd69650-85d4-492c-a0a9-4e72c32d2634/copy-manifests/0.log"
Sep 11 04:16:15.083577 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:15.083539    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q_afd69650-85d4-492c-a0a9-4e72c32d2634/manager/0.log"
Sep 11 04:16:15.433760 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:15.433680    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-778c967957-r6g9d_dc2f2e89-fba1-4798-a8e2-02a6dd7ce3d5/maas-api/0.log"
Sep 11 04:16:15.888518 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:15.888481    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xn8b2_50c51cf4-16fd-47f2-8252-2714cfdef8e8/postgres/0.log"
Sep 11 04:16:16.856596 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:16.856562    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fdrgk_b166f2c4-85b9-43a4-96c3-eedb529961a3/manager/0.log"
Sep 11 04:16:17.099233 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:17.099204    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5wbrq_f6114465-febc-4f35-83ec-a8371655abc5/kuadrant-console-plugin/0.log"
Sep 11 04:16:17.496061 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:17.496019    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-blf9s_077a4a5d-e14e-4118-93e2-57d281e98b5b/manager/0.log"
Sep 11 04:16:17.861566 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:17.861527    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cmhct8_bb4fa34b-5792-44f6-836d-9c1ae137df9e/istio-proxy/0.log"
Sep 11 04:16:18.105597 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:18.105570    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86748cc5bf-frg7j_ae641e21-ecf2-43d8-a172-e08d8da54e74/kube-auth-proxy/0.log"
Sep 11 04:16:18.350159 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:18.350128    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/1.log"
Sep 11 04:16:18.481615 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:18.481583    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k796r_b66ef199-4502-4e5c-9005-7aa9927b9b09/payload-pre-processing/0.log"
Sep 11 04:16:19.214890 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:19.214859    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-d2xxq_38144661-3419-4dd5-a3c8-2827c7d608f6/main/0.log"
Sep 11 04:16:19.448224 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:19.448191    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-dl9kz_a00dc0cd-08e0-4ba9-a250-4d9b4483a6b9/main/0.log"
Sep 11 04:16:19.599798 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:19.599766    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqnsgq_eb81cbb1-d7da-41f5-964b-d12230e72f74/main/0.log"
Sep 11 04:16:25.731430 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:25.731400    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:16:25.928148 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:25.928103    2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log" limit=1024
Sep 11 04:16:28.987556 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:28.987521    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-s5gzw_3a36fe51-02f8-4227-876f-b081ce8df153/global-pull-secret-syncer/0.log"
Sep 11 04:16:29.050712 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:29.050679    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9d52k_4bc10e86-f575-4533-94a3-911036908965/konnectivity-agent/0.log"
Sep 11 04:16:29.195368 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:29.195332    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-129.ec2.internal_85b0479989c71536ba20c64f751ffba4/haproxy/0.log"
Sep 11 04:16:34.332532 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:34.332491    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-fdrgk_b166f2c4-85b9-43a4-96c3-eedb529961a3/manager/0.log"
Sep 11 04:16:34.394450 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:34.394425    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-5wbrq_f6114465-febc-4f35-83ec-a8371655abc5/kuadrant-console-plugin/0.log"
Sep 11 04:16:34.550910 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:34.550880    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-blf9s_077a4a5d-e14e-4118-93e2-57d281e98b5b/manager/0.log"
Sep 11 04:16:36.217138 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:36.217103    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-bmlmg_ddbeddec-7069-47bb-b860-7ecfade68cda/cluster-monitoring-operator/0.log"
Sep 11 04:16:36.770542 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:36.770517    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcr24_465b9ae5-e266-4485-acb7-6e0c2fc50f0a/node-exporter/0.log"
Sep 11 04:16:36.849256 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:36.849231    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcr24_465b9ae5-e266-4485-acb7-6e0c2fc50f0a/kube-rbac-proxy/0.log"
Sep 11 04:16:36.912186 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:36.912103    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcr24_465b9ae5-e266-4485-acb7-6e0c2fc50f0a/init-textfile/0.log"
Sep 11 04:16:39.090387 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.090355    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-gdd9d_5cc12f0f-c45e-44f2-a024-97e95eebd129/networking-console-plugin/0.log"
Sep 11 04:16:39.693559 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.693527    2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"]
Sep 11 04:16:39.693928 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.693914    2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" containerName="maas-api"
Sep 11 04:16:39.693978 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.693929    2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" containerName="maas-api"
Sep 11 04:16:39.694014 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.693984    2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c9ae90-9e6a-4ebe-8c98-1e925e1b61fd" containerName="maas-api"
Sep 11 04:16:39.697433 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.697412    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.702146 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.702127    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-85rjf\"/\"kube-root-ca.crt\""
Sep 11 04:16:39.703790 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.703768    2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-85rjf\"/\"default-dockercfg-j767r\""
Sep 11 04:16:39.703903 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.703799    2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-85rjf\"/\"openshift-service-ca.crt\""
Sep 11 04:16:39.714070 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.714045    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"]
Sep 11 04:16:39.783391 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.783356    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-podres\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.783549 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.783400    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh25q\" (UniqueName: \"kubernetes.io/projected/576a8a1d-7232-41e5-9048-a9db39d797de-kube-api-access-zh25q\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.783549 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.783430    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-lib-modules\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.783549 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.783490    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-proc\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.783660 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.783550    2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-sys\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.798331 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.798309    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/2.log"
Sep 11 04:16:39.805433 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.805410    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-csgkk_44d63a06-3620-4422-aecd-45eb88efe8e9/console-operator/3.log"
Sep 11 04:16:39.884809 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884775    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-sys\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.884991 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884844    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-podres\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.884991 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884869    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-sys\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.884991 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884878    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh25q\" (UniqueName: \"kubernetes.io/projected/576a8a1d-7232-41e5-9048-a9db39d797de-kube-api-access-zh25q\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.884991 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884922    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-lib-modules\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.884991 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.884963    2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-proc\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.885197 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.885027    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-proc\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.885197 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.885035    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-podres\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.885197 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.885061    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/576a8a1d-7232-41e5-9048-a9db39d797de-lib-modules\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:39.897507 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:39.897486    2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh25q\" (UniqueName: \"kubernetes.io/projected/576a8a1d-7232-41e5-9048-a9db39d797de-kube-api-access-zh25q\") pod \"perf-node-gather-daemonset-lszkd\" (UID: \"576a8a1d-7232-41e5-9048-a9db39d797de\") " pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:40.007058 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:40.006982    2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:40.277928 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:40.277901    2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"]
Sep 11 04:16:40.279329 ip-10-0-139-129 kubenswrapper[2553]: W0911 04:16:40.279298    2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod576a8a1d_7232_41e5_9048_a9db39d797de.slice/crio-93b9cf800559af207558b156151a6fc81bf1919f3f4606bf9f2f9b984a21c60f WatchSource:0}: Error finding container 93b9cf800559af207558b156151a6fc81bf1919f3f4606bf9f2f9b984a21c60f: Status 404 returned error can't find the container with id 93b9cf800559af207558b156151a6fc81bf1919f3f4606bf9f2f9b984a21c60f
Sep 11 04:16:40.994116 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:40.994083    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd" event={"ID":"576a8a1d-7232-41e5-9048-a9db39d797de","Type":"ContainerStarted","Data":"af623689927d16d8ff37ef675173922ff65d91179c3263ebd88591be75ab14a9"}
Sep 11 04:16:40.994286 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:40.994133    2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd" event={"ID":"576a8a1d-7232-41e5-9048-a9db39d797de","Type":"ContainerStarted","Data":"93b9cf800559af207558b156151a6fc81bf1919f3f4606bf9f2f9b984a21c60f"}
Sep 11 04:16:40.994286 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:40.994188    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:41.014271 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:41.014218    2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd" podStartSLOduration=2.014203235 podStartE2EDuration="2.014203235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:16:41.011385282 +0000 UTC m=+2190.654175444" watchObservedRunningTime="2026-09-11 04:16:41.014203235 +0000 UTC m=+2190.656993398"
Sep 11 04:16:42.216635 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:42.216604    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pfwn5_344210ec-7df8-4263-a6ca-3fda63110b03/dns/0.log"
Sep 11 04:16:42.238647 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:42.238620    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pfwn5_344210ec-7df8-4263-a6ca-3fda63110b03/kube-rbac-proxy/0.log"
Sep 11 04:16:42.372764 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:42.372736    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7c4sh_c64109e2-8108-4841-bac6-5117f7bf60a1/dns-node-resolver/0.log"
Sep 11 04:16:42.985238 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:42.985206    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jgkcv_7dba23ae-c972-4567-9926-d61a07546e4d/node-ca/0.log"
Sep 11 04:16:44.020279 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:44.020240    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cmhct8_bb4fa34b-5792-44f6-836d-9c1ae137df9e/istio-proxy/0.log"
Sep 11 04:16:44.332717 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:44.332633    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86748cc5bf-frg7j_ae641e21-ecf2-43d8-a172-e08d8da54e74/kube-auth-proxy/0.log"
Sep 11 04:16:44.393831 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:44.393786    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/0.log"
Sep 11 04:16:44.408316 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:44.408294    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/1.log"
Sep 11 04:16:44.457084 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:44.457060    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k796r_b66ef199-4502-4e5c-9005-7aa9927b9b09/payload-pre-processing/0.log"
Sep 11 04:16:45.512994 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:45.512967    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wjcjz_2396b115-42d4-482c-849c-cddb2ea07ba5/serve-healthcheck-canary/0.log"
Sep 11 04:16:46.330702 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:46.330677    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qbbh_0a3fef89-bb3c-46b6-b784-6e4f91cb8e43/kube-rbac-proxy/0.log"
Sep 11 04:16:46.364069 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:46.364038    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qbbh_0a3fef89-bb3c-46b6-b784-6e4f91cb8e43/exporter/0.log"
Sep 11 04:16:46.401266 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:46.401227    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qbbh_0a3fef89-bb3c-46b6-b784-6e4f91cb8e43/extractor/0.log"
Sep 11 04:16:47.008643 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:47.008615    2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-85rjf/perf-node-gather-daemonset-lszkd"
Sep 11 04:16:49.272799 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:49.272764    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hhtzd_e0aacd83-19e1-4965-a283-c82b75a7f1c3/manager/0.log"
Sep 11 04:16:49.531913 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:49.531884    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-b9pcn_a41e9d5b-4228-4516-9117-a9282bd84e5c/manager/0.log"
Sep 11 04:16:49.634950 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:49.634916    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-66fc969cd5-pkqcp_54247ecb-72d2-4423-8cd4-b947785e2355/manager/0.log"
Sep 11 04:16:49.672180 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:49.672150    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6jngc_01c0b92a-72df-401d-be77-d5656b6accaa/server/0.log"
Sep 11 04:16:49.795012 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:49.794980    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9fscw_e1290a39-9e24-4ebf-8d61-70e9bbad7590/manager/0.log"
Sep 11 04:16:50.147390 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.147359    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cmhct8_bb4fa34b-5792-44f6-836d-9c1ae137df9e/istio-proxy/0.log"
Sep 11 04:16:50.148129 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.148116    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q_afd69650-85d4-492c-a0a9-4e72c32d2634/manager/0.log"
Sep 11 04:16:50.220583 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.220560    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cf4d46bbf-vtf4q_afd69650-85d4-492c-a0a9-4e72c32d2634/copy-manifests/0.log"
Sep 11 04:16:50.483496 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.483407    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86748cc5bf-frg7j_ae641e21-ecf2-43d8-a172-e08d8da54e74/kube-auth-proxy/0.log"
Sep 11 04:16:50.531589 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.531559    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/0.log"
Sep 11 04:16:50.547186 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.547157    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/1.log"
Sep 11 04:16:50.593049 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:50.593019    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k796r_b66ef199-4502-4e5c-9005-7aa9927b9b09/payload-pre-processing/0.log"
Sep 11 04:16:51.625050 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:51.624975    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rs2jk_064de959-1713-4003-98c8-aa2cf405ec34/openshift-lws-operator/0.log"
Sep 11 04:16:52.294110 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:52.294079    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cmhct8_bb4fa34b-5792-44f6-836d-9c1ae137df9e/istio-proxy/0.log"
Sep 11 04:16:52.624589 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:52.624505    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86748cc5bf-frg7j_ae641e21-ecf2-43d8-a172-e08d8da54e74/kube-auth-proxy/0.log"
Sep 11 04:16:52.703175 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:52.703146    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/0.log"
Sep 11 04:16:52.715822 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:52.715797    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-5zqtx_6d9335c5-6a0b-4020-98de-d4eaafdfba1b/istio-proxy/1.log"
Sep 11 04:16:52.765787 ip-10-0-139-129 kubenswrapper[2553]: I0911 04:16:52.765758    2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k796r_b66ef199-4502-4e5c-9005-7aa9927b9b09/payload-pre-processing/0.log"