Jul 17 09:16:22.226498 ip-10-0-139-140 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 09:16:22.226511 ip-10-0-139-140 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 09:16:22.226520 ip-10-0-139-140 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 09:16:22.226860 ip-10-0-139-140 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 09:16:32.475530 ip-10-0-139-140 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 09:16:32.475549 ip-10-0-139-140 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a2f6d66e2daf460b92e42b87ea41fe09 --
Jul 17 09:18:05.119830 ip-10-0-139-140 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:05.560449 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.557659    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562216    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562227    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562231    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562234    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562237    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562240    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562243    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562246    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562249    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562252    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562255    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562257    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562260    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562263    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562266    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562269    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562271    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562274    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.579870 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562278    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562280    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562283    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562286    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562293    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562296    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562299    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562301    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562304    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562307    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562309    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562312    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562315    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562317    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562320    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562323    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562325    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562328    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562330    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.593013 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562333    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562336    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562338    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562342    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562344    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562347    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562350    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562352    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562355    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562360    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562364    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562367    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562370    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562372    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562375    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562377    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562380    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562383    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562385    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.595204 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562388    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562391    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562393    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562396    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562399    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562402    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562404    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562408    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562410    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562413    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562416    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562418    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562421    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562423    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562426    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562429    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562432    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562434    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562437    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562440    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562442    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.611660 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562445    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562447    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562450    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562452    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562456    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562460    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562463    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562465    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562469    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562474    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562477    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562480    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562482    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562485    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562488    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562491    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562494    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562496    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562499    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.613921 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562502    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562505    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.562508    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563176    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563182    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563187    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563191    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563195    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563198    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563201    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563204    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563207    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563209    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563212    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563215    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563218    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563221    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563223    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563226    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.667116 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563229    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.649357 ip-10-0-139-140 systemd[1]: Started Kubernetes Kubelet.
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563232    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563234    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563237    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563239    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563242    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563244    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563247    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563250    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563252    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563254    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563257    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563260    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563262    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563265    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563270    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563272    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563275    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563278    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563280    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.669327 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563283    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563286    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563289    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563292    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563294    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563297    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563301    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563305    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563308    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563311    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563313    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563316    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563319    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563322    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563324    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563327    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563330    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563333    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563335    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.680083 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563339    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563342    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563345    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563347    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563350    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563353    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563355    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563358    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563361    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563364    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563367    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563370    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563372    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563374    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563377    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563381    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563385    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563389    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563392    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563394    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.682759 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563397    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563400    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563403    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563405    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563408    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563410    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563413    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563415    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563418    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563421    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563423    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563426    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563428    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563431    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563433    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563436    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563439    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563441    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563444    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563446    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.689107 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563450    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563453    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563456    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563458    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563890    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563897    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563900    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563904    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563907    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563910    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563914    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563916    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563919    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563922    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563925    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563928    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563931    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563934    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563936    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563939    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.849319 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563941    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563944    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563947    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563949    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563952    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563955    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563957    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563960    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563963    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563965    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563968    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563970    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563975    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563977    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563980    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563983    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563985    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563988    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563990    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563993    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.868636 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563995    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.563998    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564002    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564005    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564008    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564011    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564013    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564016    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564018    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564021    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564023    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564026    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564028    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564030    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564033    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564035    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564038    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564041    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564043    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564046    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.869372 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564048    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564051    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564053    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564056    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564059    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564062    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564064    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564067    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564071    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564074    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564077    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564080    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564083    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564085    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564090    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564092    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564095    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564097    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564100    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564102    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.871267 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564105    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564107    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564111    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564114    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564116    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564119    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564121    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564124    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564128    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564131    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564135    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564137    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564140    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564142    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564145    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564147    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564151    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564153    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564156    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.872038 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564158    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564161    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564165    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564168    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564229    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564235    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564246    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564252    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564256    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564259    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564263    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564267    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564270    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564273    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564277    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564280    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564283    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564286    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564288    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564291    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564294    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564297    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564301    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 17 09:18:05.872747 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564304    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564307    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564309    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564312    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564316    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564319    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564323    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564326    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564329    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564332    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564335    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564338    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564341    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564345    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564348    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564351    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564355    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564358    2575 flags.go:64] FLAG: --enable-server="true"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564361    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564366    2575 flags.go:64] FLAG: --event-burst="100"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564369    2575 flags.go:64] FLAG: --event-qps="50"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564372    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564375    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564378    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 17 09:18:05.877838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564381    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564384    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564387    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564390    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564392    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564395    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564398    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564401    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564404    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564407    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564409    2575 flags.go:64] FLAG: --feature-gates=""
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564413    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564416    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564419    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564423    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564426    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564429    2575 flags.go:64] FLAG: --help="false"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564431    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564435    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564446    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564449    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564452    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564455    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564458    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 09:18:05.879852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564463    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564465    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564468    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564472    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564475    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564478    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564480    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564483    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564486    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564489    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564492    2575 flags.go:64] FLAG: --lock-file=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564495    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564497    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564500    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564505    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564509    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564511    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564514    2575 flags.go:64] FLAG: --logging-format="text"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564517    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564520    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564523    2575 flags.go:64] FLAG: --manifest-url=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564525    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564546    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564552    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564558    2575 flags.go:64] FLAG: --max-pods="110"
Jul 17 09:18:05.884667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564563    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564566    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564569    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564572    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564575    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564578    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564581    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564589    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564592    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564595    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564598    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564602    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564607    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564628    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564645    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564650    2575 flags.go:64] FLAG: --port="10250"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564654    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564658    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dbe32a7335fd119c"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564661    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564665    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564669    2575 flags.go:64] FLAG: --register-node="true"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564672    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564682    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564685    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 17 09:18:05.886710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564688    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564691    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564695    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564698    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564701    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564704    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564707    2575 flags.go:64] FLAG: --runonce="false"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564710    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564714    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564718    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564721    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564724    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564727    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564730    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564734    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564739    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564743    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564746    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564749    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564752    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564769    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564773    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564783    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564786    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564789    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 09:18:05.889583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564795    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564798    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564801    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564804    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564807    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564810    2575 flags.go:64] FLAG: --v="2"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564814    2575 flags.go:64] FLAG: --version="false"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564817    2575 flags.go:64] FLAG: --vmodule=""
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564821    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.564824    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564945    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564949    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564952    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564956    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564960    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564962    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564965    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564968    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564970    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564973    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564976    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564978    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.890958 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564983    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564985    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564988    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564991    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564993    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.564997    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565000    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565002    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565005    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565008    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565012    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565015    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565018    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565020    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565023    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565026    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565029    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565031    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565034    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565036    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.891951 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565039    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565041    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565044    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565046    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565049    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565052    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565055    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565057    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565060    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565062    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565065    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565067    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565071    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565074    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565077    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565079    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565082    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565086    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565088    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565091    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.893236 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565093    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565097    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565099    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565102    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565105    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565108    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565110    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565114    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565117    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565120    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565123    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565125    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565128    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565130    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565134    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565138    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565141    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565143    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565146    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.894197 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565148    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565151    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565154    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565156    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565158    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565162    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565165    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565167    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565170    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565172    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565176    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565179    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565181    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565184    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565187    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565191    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565193    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565196    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565198    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565201    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.895132 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565203    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565206    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565208    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565211    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565213    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565216    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565219    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.565221    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.565897    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577142    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577156    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577174    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577183    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577235    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577240    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.896053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577244    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577247    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577250    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577253    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577256    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577258    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577261    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577264    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577266    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577269    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577272    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577274    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577277    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577280    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577283    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577285    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577288    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577290    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577293    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577296    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.896937 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577298    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577301    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577304    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577307    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577309    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577312    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577314    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577317    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577319    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577322    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577324    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577327    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577329    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577332    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577334    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577337    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577340    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577343    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577345    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577348    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.897944 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577350    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577353    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577356    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577358    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577361    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577364    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577366    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577369    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577373    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577377    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577380    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577383    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577385    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577388    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577391    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577394    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577396    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577400    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577404    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.899194 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577407    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577409    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577412    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577415    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577418    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577420    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577423    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577425    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577428    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577430    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577433    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577436    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577438    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577441    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577443    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577446    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577449    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577451    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577454    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577458    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.900120 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577460    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577463    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577466    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577469    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577471    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577474    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577476    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577479    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577481    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577484    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577487    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577489    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577491    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577494    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577496    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577503    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577507    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.901034 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577511    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577516    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577642    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577647    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577650    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577653    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577657    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577660    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577663    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577666    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577669    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577672    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577674    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577677    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577679    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:05.903168 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577683    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577685    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577688    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577691    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577694    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577696    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577699    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577701    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577704    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577706    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577709    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577711    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577713    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577716    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577718    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577720    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577725    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577728    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577730    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577733    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:05.904064 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577735    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577737    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577740    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577742    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577745    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577747    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577750    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577752    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577754    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577757    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577759    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577762    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577764    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577768    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577771    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577774    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577776    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577779    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577782    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577784    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:05.906604 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577787    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577789    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577792    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577794    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577798    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577801    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577804    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577806    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577810    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577812    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577815    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577818    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577820    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577823    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577825    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577828    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577830    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577833    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577837    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:05.907514 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577840    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577843    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577846    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577849    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577852    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577854    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577857    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577861    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577864    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577866    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577869    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577871    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577874    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577876    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577879    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577881    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577884    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577886    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577889    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577891    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:05.909790 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577894    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577897    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577900    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577903    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577905    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577909    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:05.577912    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.577917    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.578013    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.580389    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.582147    2575 server.go:1019] "Starting client certificate rotation"
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.582254    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.582300    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:18:05.910720 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.585017    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.591931    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.607368    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.611358    2575 log.go:25] "Validated CRI v1 image API"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.612489    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.613821    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.615187    2575 fs.go:135] Filesystem UUIDs: map[32d274a0-b819-4d4f-8912-fe65843d6d20:/dev/nvme0n1p3 4a1cdad5-54fd-47d8-ba9a-b3548aa77d3d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 17 09:18:05.911613 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.615203    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.619258    2575 manager.go:216] Machine: {Timestamp:2026-07-17 09:18:05.618806672 +0000 UTC m=+0.370562961 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114454 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22a8934e8dc738c58ebb47987b8d1a SystemUUID:ec22a893-4e8d-c738-c58e-bb47987b8d1a BootID:a2f6d66e-2daf-460b-92e4-2b87ea41fe09 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:7e:ef:2f:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:7e:ef:2f:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:01:58:7b:d2:0e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.619345    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.619437    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.623121    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.623145    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-140.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.624657    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.624668    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.624680    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.624694    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.626573    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631712    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631817    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631842    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631851    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631864    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631873    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.631881    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 09:18:05.999341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.632187    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.632208    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.636038    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vgcsh"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.638228    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.642388    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vgcsh"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.643177    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.643210    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.643251    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.643349    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644630    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644650    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644657    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644663    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644672    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644682    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644691    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644696    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644704    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644709    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644720    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.644726    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.645602    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.645611    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.648598    2575 server.go:1268] "Started kubelet"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.648658    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.648671    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.648724    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.650164    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.662476    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.662793    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.670131    2575 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-140.ec2.internal" not found
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.674797    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.674812    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.674922    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675563    2575 factory.go:55] Registering systemd factory
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675578    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675634    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675656    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675674    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675777    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675787    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675786    2575 factory.go:153] Registering CRI-O factory
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675814    2575 factory.go:223] Registration of the crio container factory successfully
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675866    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675890    2575 factory.go:103] Registering Raw factory
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.675905    2575 manager.go:1196] Started watching for new ooms in manager
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.676895    2575 manager.go:349] Starting recovery of all containers
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.677628    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:06.001465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.683107    2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-140.ec2.internal\" not found" node="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.685714    2575 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-140.ec2.internal" not found
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.693666    2575 manager.go:354] Recovery completed
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.697506    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.699739    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.699765    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.699776    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.700233    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.700244    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.700255    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.700262    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.701002    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.701026    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.701035    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.702523    2575 policy_none.go:49] "None policy: Start"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.702553    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.702567    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.704575    2575 policy_none.go:47] "Start"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728231    2575 manager.go:337] "Starting Device Plugin manager"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.728264    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728276    2575 server.go:87] "Starting device plugin registration server"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728489    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728500    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728614    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728680    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.728689    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.729122    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.729169    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.749107    2575 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-139-140.ec2.internal" not found
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.785681    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.786786    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.786828    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.786848    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.786880    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.789756    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.828760    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.829579    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.829606    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.829616    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.829648    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.838928    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.838943    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-140.ec2.internal\": node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.855629    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.887049    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"]
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.887133    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.888010    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.888040    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.003785 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.888050    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.890333    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.890445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.890475    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891346    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891374    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891386    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891402    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891424    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.891437    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.893655    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.893675    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.894292    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.894318    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.894331    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:05.956476    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.977751    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.977775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.005683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:05.977790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/986b8db4c072b1a4199cd963bf9e6ab5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-140.ec2.internal\" (UID: \"986b8db4c072b1a4199cd963bf9e6ab5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.057299    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20a58c6089fe64c4052e1369fd869a24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal\" (UID: \"20a58c6089fe64c4052e1369fd869a24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/986b8db4c072b1a4199cd963bf9e6ab5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-140.ec2.internal\" (UID: \"986b8db4c072b1a4199cd963bf9e6ab5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.111221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.078803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/986b8db4c072b1a4199cd963bf9e6ab5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-140.ec2.internal\" (UID: \"986b8db4c072b1a4199cd963bf9e6ab5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:06.222614 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.157912    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.330775 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.258762    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.446056 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.359172    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.464339 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.459617    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:06.562110 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.560035    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.581308    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.581426    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:06.581447    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:06.581485    2575 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aa942d10396c64faba8e5a6704bb6634-b80e851cd7498e5c.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-139-140.ec2.internal.18c3091a3f028c1b\": read tcp 10.0.139.140:56040->98.87.90.36:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-139-140.ec2.internal.18c3091a3f028c1b  default   5803 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-140.ec2.internal,UID:ip-10-0-139-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-140.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-140.ec2.internal,},FirstTimestamp:2026-07-17 09:18:05 +0000 UTC,LastTimestamp:2026-07-17 09:18:05.89144105 +0000 UTC m=+0.643197322,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-140.ec2.internal,}"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.008476    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 09:13:06 +0000 UTC" deadline="2028-02-12 17:02:36.638935813 +0000 UTC"
Jul 17 09:18:08.014218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.008497    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13807h44m28.630441754s"
Jul 17 09:18:08.022485 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.020601    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:08.121705 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.121683    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:08.142523 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.142500    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:08.147712 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.147692    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-140.ec2.internal\" not found" node="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.150395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.150381    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.151464 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.151449    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-140.ec2.internal\" not found" node="ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.153481 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.153469    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.222605 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.222580    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:08.323078 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:08.323054    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-140.ec2.internal\" not found"
Jul 17 09:18:08.340169 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.340151    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:08.435193 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.435139    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.446627 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.446610    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:18:08.447493 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.447481    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal"
Jul 17 09:18:08.456536 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.456522    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:18:08.809441 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:08.809394    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986b8db4c072b1a4199cd963bf9e6ab5.slice/crio-548228f4b162f8dfb52f518d4cf8646073dcc31452aeaa9d01417ad5ca3441bf WatchSource:0}: Error finding container 548228f4b162f8dfb52f518d4cf8646073dcc31452aeaa9d01417ad5ca3441bf: Status 404 returned error can't find the container with id 548228f4b162f8dfb52f518d4cf8646073dcc31452aeaa9d01417ad5ca3441bf
Jul 17 09:18:08.810167 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:08.810144    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a58c6089fe64c4052e1369fd869a24.slice/crio-aca6f05b79cc5c49628a0197918e97a9a93f1963ee7055b81cfdb2263efd6890 WatchSource:0}: Error finding container aca6f05b79cc5c49628a0197918e97a9a93f1963ee7055b81cfdb2263efd6890: Status 404 returned error can't find the container with id aca6f05b79cc5c49628a0197918e97a9a93f1963ee7055b81cfdb2263efd6890
Jul 17 09:18:08.813551 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.813535    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:18:08.993210 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:08.993185    2575 apiserver.go:52] "Watching apiserver"
Jul 17 09:18:09.002950 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.002928    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 09:18:09.003908 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.003882    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-5wq6g","openshift-multus/multus-6nln8","openshift-multus/network-metrics-daemon-6pxs2","openshift-network-diagnostics/network-check-target-zql5k","openshift-ovn-kubernetes/ovnkube-node-lhrpg","kube-system/konnectivity-agent-nbvvs","kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal","openshift-cluster-node-tuning-operator/tuned-6sldl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal","openshift-multus/multus-additional-cni-plugins-ssbbs","openshift-network-operator/iptables-alerter-s2fl6","kube-system/global-pull-secret-syncer-tl6xw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm","openshift-dns/node-resolver-zkm7q"]
Jul 17 09:18:09.007622 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.007606    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.009664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.009646    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.010456 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010436    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.010610 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010595    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 09:18:09.010740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010722    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 09:18:09.010740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010734    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 09:18:09.010875 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010779    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.010875 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.010824    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b5s7r\""
Jul 17 09:18:09.011787 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.011771    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.011871 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.011847    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:09.011997 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.011982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.012186 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.012168    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-shd42\""
Jul 17 09:18:09.012278 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.012193    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 09:18:09.012869 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.012852    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.014134 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.014116    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:09.014205 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.014187    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:09.016513 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.016498    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.018702 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.018683    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.019130 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019112    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 09:18:09.019257 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019140    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 09:18:09.019257 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019201    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 09:18:09.019416 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019366    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.019416 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019390    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 09:18:09.019514 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.019391    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q6q7c\""
Jul 17 09:18:09.020235 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.020218    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.020814 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.020797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.021396 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.021372    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4sbq8\""
Jul 17 09:18:09.021496 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.021471    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 09:18:09.021496 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.021488    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 09:18:09.022927 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.022914    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.023076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.023061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.023243 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.023223    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.023348 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.023338    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h52bc\""
Jul 17 09:18:09.025555 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.025530    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.025644 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.025539    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.025644 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.025560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v5njt\""
Jul 17 09:18:09.026675 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.026657    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.028805 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.028791    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 09:18:09.028805 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.028799    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5qr6z\""
Jul 17 09:18:09.028937 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.028907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.031030 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.030993    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 09:18:09.031266 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.031252    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.031359 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.031329    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.031634 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.031614    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.031719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.031686    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6mzm5\""
Jul 17 09:18:09.033561 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.033544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.033651 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.033580    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:18:09.033651 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.033581    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mpnxw\""
Jul 17 09:18:09.033651 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.033603    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:09.033788 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.033736    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 09:18:09.033857 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.033843    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 09:18:09.034807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.034706    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 09:18:09.036591 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.036574    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 09:18:09.046218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046198    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-cnibin\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.046291 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046224    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.046291 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-system-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.046291 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-node-log\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046291 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046270    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-netd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-config\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmrfd\" (UniqueName: \"kubernetes.io/projected/ea774c5b-e13d-4cd3-af7a-c46138494b3e-kube-api-access-gmrfd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cni-binary-copy\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-conf-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-systemd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-var-lib-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046469 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046457    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-agent-certs\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-modprobe-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046513    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-tuned\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5da12a08-c074-4cad-b523-fffdc5fa3002-serviceca\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-ovn\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-lib-modules\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046587    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbjx\" (UniqueName: \"kubernetes.io/projected/c9bb0622-72c9-4424-a5e1-e83111a57c83-kube-api-access-xwbjx\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046602    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-socket-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-sys-fs\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046654    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-kubelet\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046670    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-var-lib-kubelet\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-systemd-units\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.046740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-netns\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9bb0622-72c9-4424-a5e1-e83111a57c83-iptables-alerter-script\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046766    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-device-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-multus-certs\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046799    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-etc-kubernetes\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zx6w\" (UniqueName: \"kubernetes.io/projected/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-kube-api-access-7zx6w\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-k8s-cni-cncf-io\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046884    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-script-lib\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-conf\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-systemd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5da12a08-c074-4cad-b523-fffdc5fa3002-host\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-hosts-file\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-os-release\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.046993    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-bin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-daemon-config\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047030    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-konnectivity-ca\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047096    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-hostroot\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-env-overrides\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79pmr\" (UniqueName: \"kubernetes.io/projected/0badef41-1489-4263-a9b7-489f7421d5c1-kube-api-access-79pmr\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysconfig\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047186    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047227    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-kubelet\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-bin\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047288    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-sys\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047367    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-kubelet-config\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khk2z\" (UniqueName: \"kubernetes.io/projected/0da71ae0-35dc-4281-b2fa-8bbea3556743-kube-api-access-khk2z\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-netns\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.047741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0badef41-1489-4263-a9b7-489f7421d5c1-ovn-node-metrics-cert\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-host\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047516    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-tmp\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z4kv\" (UniqueName: \"kubernetes.io/projected/cc5779d7-9440-4f6f-b8e1-4ca12489f555-kube-api-access-5z4kv\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8mvt\" (UniqueName: \"kubernetes.io/projected/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-kube-api-access-g8mvt\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047629    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-kubernetes\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047664    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9bb0622-72c9-4424-a5e1-e83111a57c83-host-slash\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-system-cni-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047741    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-os-release\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvmf6\" (UniqueName: \"kubernetes.io/projected/8b42ab0d-e694-4cf6-9054-819cf82b1790-kube-api-access-wvmf6\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-etc-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047833    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-slash\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.048290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v85v\" (UniqueName: \"kubernetes.io/projected/5da12a08-c074-4cad-b523-fffdc5fa3002-kube-api-access-8v85v\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047893    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-registration-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047907    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cnibin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-socket-dir-parent\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047935    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-log-socket\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047948    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-dbus\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.047991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-tmp-dir\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.048022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-multus\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.048817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.048052    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-run\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.069038 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.069021    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5zzzk"
Jul 17 09:18:09.077493 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.077476    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5zzzk"
Jul 17 09:18:09.116164 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.116142    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:09.148426 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbjx\" (UniqueName: \"kubernetes.io/projected/c9bb0622-72c9-4424-a5e1-e83111a57c83-kube-api-access-xwbjx\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.148426 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-socket-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.148573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148445    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-sys-fs\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.148573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-sys-fs\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.148672 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148591    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-kubelet\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148672 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148616    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-socket-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.148672 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-var-lib-kubelet\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-var-lib-kubelet\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-kubelet\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148707    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-systemd-units\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-netns\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9bb0622-72c9-4424-a5e1-e83111a57c83-iptables-alerter-script\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148747    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-systemd-units\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-device-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148775    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-multus-certs\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-netns\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.148815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148797    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-etc-kubernetes\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148817    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-device-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148862    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-etc-selinux\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx6w\" (UniqueName: \"kubernetes.io/projected/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-kube-api-access-7zx6w\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-multus-certs\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148890    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-k8s-cni-cncf-io\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-etc-kubernetes\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-k8s-cni-cncf-io\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148963    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.148990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-script-lib\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-conf\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-systemd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149055    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5da12a08-c074-4cad-b523-fffdc5fa3002-host\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-hosts-file\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5da12a08-c074-4cad-b523-fffdc5fa3002-host\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149112    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-os-release\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.149262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149136    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-bin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-daemon-config\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-konnectivity-ca\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-conf\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-hostroot\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149287    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-env-overrides\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79pmr\" (UniqueName: \"kubernetes.io/projected/0badef41-1489-4263-a9b7-489f7421d5c1-kube-api-access-79pmr\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c9bb0622-72c9-4424-a5e1-e83111a57c83-iptables-alerter-script\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149390    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-bin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149424    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-hosts-file\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-systemd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-hostroot\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149461    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.149484    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysconfig\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149810    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-env-overrides\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149636    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-script-lib\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysconfig\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-os-release\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.149808    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:09.649785942 +0000 UTC m=+3.041778553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149899    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149947    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-daemon-config\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149950    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-konnectivity-ca\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-kubelet\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.149983    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149995    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150024    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-kubelet\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150027    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-bin\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.150046    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:09.650030137 +0000 UTC m=+3.042022732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.149998    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-bin\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-sys\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.150800 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-kubelet-config\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khk2z\" (UniqueName: \"kubernetes.io/projected/0da71ae0-35dc-4281-b2fa-8bbea3556743-kube-api-access-khk2z\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-sys\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-netns\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150177    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0badef41-1489-4263-a9b7-489f7421d5c1-ovn-node-metrics-cert\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-kubelet-config\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-run-netns\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-host\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-tmp\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150406    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150432    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z4kv\" (UniqueName: \"kubernetes.io/projected/cc5779d7-9440-4f6f-b8e1-4ca12489f555-kube-api-access-5z4kv\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-host\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150457    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8mvt\" (UniqueName: \"kubernetes.io/projected/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-kube-api-access-g8mvt\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150482    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-kubernetes\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150496    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9bb0622-72c9-4424-a5e1-e83111a57c83-host-slash\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-system-cni-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-os-release\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.151574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150598    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150601    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c9bb0622-72c9-4424-a5e1-e83111a57c83-host-slash\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvmf6\" (UniqueName: \"kubernetes.io/projected/8b42ab0d-e694-4cf6-9054-819cf82b1790-kube-api-access-wvmf6\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150692    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-os-release\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-system-cni-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-etc-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150749    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-etc-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.150787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-kubernetes\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-slash\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v85v\" (UniqueName: \"kubernetes.io/projected/5da12a08-c074-4cad-b523-fffdc5fa3002-kube-api-access-8v85v\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-cni-binary-copy\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-slash\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151550    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-registration-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.152473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151636    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cnibin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-socket-dir-parent\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151720    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0da71ae0-35dc-4281-b2fa-8bbea3556743-registration-dir\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-socket-dir-parent\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151765    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-log-socket\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151785    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cnibin\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-dbus\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151854    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-log-socket\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151863    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-tmp-dir\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-multus\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151929    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-run\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7849ad77-4297-4d63-97eb-4f60039e4a1b-dbus\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-cnibin\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.151994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-host-var-lib-cni-multus\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-system-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-node-log\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.153252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152048    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-system-cni-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152064    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152058    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal" event={"ID":"986b8db4c072b1a4199cd963bf9e6ab5","Type":"ContainerStarted","Data":"548228f4b162f8dfb52f518d4cf8646073dcc31452aeaa9d01417ad5ca3441bf"}
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152089    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-netd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-config\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152121    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-cnibin\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152092    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-run\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152138    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmrfd\" (UniqueName: \"kubernetes.io/projected/ea774c5b-e13d-4cd3-af7a-c46138494b3e-kube-api-access-gmrfd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152149    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-node-log\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152161    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cni-binary-copy\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152168    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-cni-netd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152176    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-tmp-dir\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-conf-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-systemd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-var-lib-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b42ab0d-e694-4cf6-9054-819cf82b1790-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-agent-certs\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-modprobe-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152293    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152354    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-tuned\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152363    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-var-lib-openvswitch\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5da12a08-c074-4cad-b523-fffdc5fa3002-serviceca\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152399    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-ovn\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152403    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-multus-conf-dir\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-lib-modules\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-ovn\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-modprobe-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-lib-modules\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152606    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-sysctl-d\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152646    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0badef41-1489-4263-a9b7-489f7421d5c1-run-systemd\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152703    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0badef41-1489-4263-a9b7-489f7421d5c1-ovnkube-config\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152721    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-cni-binary-copy\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152824    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b42ab0d-e694-4cf6-9054-819cf82b1790-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.152938    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5da12a08-c074-4cad-b523-fffdc5fa3002-serviceca\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.154023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal" event={"ID":"20a58c6089fe64c4052e1369fd869a24","Type":"ContainerStarted","Data":"aca6f05b79cc5c49628a0197918e97a9a93f1963ee7055b81cfdb2263efd6890"}
Jul 17 09:18:09.154944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.154430    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0badef41-1489-4263-a9b7-489f7421d5c1-ovn-node-metrics-cert\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.155457 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.154451    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-tmp\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.155457 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.154742    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ea774c5b-e13d-4cd3-af7a-c46138494b3e-etc-tuned\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.155457 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.155100    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bb36912c-324d-411b-a59b-9aa6c9bcd7b2-agent-certs\") pod \"konnectivity-agent-nbvvs\" (UID: \"bb36912c-324d-411b-a59b-9aa6c9bcd7b2\") " pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.157812 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.157793    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:09.157936 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.157818    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:09.157936 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.157832    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:09.157936 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.157883    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:09.657866339 +0000 UTC m=+3.049858979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:09.160049 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zx6w\" (UniqueName: \"kubernetes.io/projected/c8684fe2-2823-4aa4-bbbd-1ebcc2239523-kube-api-access-7zx6w\") pod \"node-resolver-zkm7q\" (UID: \"c8684fe2-2823-4aa4-bbbd-1ebcc2239523\") " pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.160474 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khk2z\" (UniqueName: \"kubernetes.io/projected/0da71ae0-35dc-4281-b2fa-8bbea3556743-kube-api-access-khk2z\") pod \"aws-ebs-csi-driver-node-m5zmm\" (UID: \"0da71ae0-35dc-4281-b2fa-8bbea3556743\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.160585 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160515    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbjx\" (UniqueName: \"kubernetes.io/projected/c9bb0622-72c9-4424-a5e1-e83111a57c83-kube-api-access-xwbjx\") pod \"iptables-alerter-s2fl6\" (UID: \"c9bb0622-72c9-4424-a5e1-e83111a57c83\") " pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.160682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160585    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z4kv\" (UniqueName: \"kubernetes.io/projected/cc5779d7-9440-4f6f-b8e1-4ca12489f555-kube-api-access-5z4kv\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.160682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvmf6\" (UniqueName: \"kubernetes.io/projected/8b42ab0d-e694-4cf6-9054-819cf82b1790-kube-api-access-wvmf6\") pod \"multus-additional-cni-plugins-ssbbs\" (UID: \"8b42ab0d-e694-4cf6-9054-819cf82b1790\") " pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.160682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.160610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8mvt\" (UniqueName: \"kubernetes.io/projected/4aeb1cf2-5fa8-448d-8aed-cfb3f875f063-kube-api-access-g8mvt\") pod \"multus-6nln8\" (UID: \"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063\") " pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.161053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.161034    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmrfd\" (UniqueName: \"kubernetes.io/projected/ea774c5b-e13d-4cd3-af7a-c46138494b3e-kube-api-access-gmrfd\") pod \"tuned-6sldl\" (UID: \"ea774c5b-e13d-4cd3-af7a-c46138494b3e\") " pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.161755 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.161740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v85v\" (UniqueName: \"kubernetes.io/projected/5da12a08-c074-4cad-b523-fffdc5fa3002-kube-api-access-8v85v\") pod \"node-ca-5wq6g\" (UID: \"5da12a08-c074-4cad-b523-fffdc5fa3002\") " pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.161809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.161744    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79pmr\" (UniqueName: \"kubernetes.io/projected/0badef41-1489-4263-a9b7-489f7421d5c1-kube-api-access-79pmr\") pod \"ovnkube-node-lhrpg\" (UID: \"0badef41-1489-4263-a9b7-489f7421d5c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.317073 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.316981    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ssbbs"
Jul 17 09:18:09.323035 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.323010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm"
Jul 17 09:18:09.330561 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.330519    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b42ab0d_e694_4cf6_9054_819cf82b1790.slice/crio-530144d3efa0900dc72501a09664ec8db891cfd2b2a65c06ced926f8609bd89e WatchSource:0}: Error finding container 530144d3efa0900dc72501a09664ec8db891cfd2b2a65c06ced926f8609bd89e: Status 404 returned error can't find the container with id 530144d3efa0900dc72501a09664ec8db891cfd2b2a65c06ced926f8609bd89e
Jul 17 09:18:09.332536 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.332515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:09.338145 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.338122    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6sldl"
Jul 17 09:18:09.343003 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.342974    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0badef41_1489_4263_a9b7_489f7421d5c1.slice/crio-aacdb4e3a9f9938b9dcbad3cb313282c82b76100196f3a1ecd81fd87d58d55eb WatchSource:0}: Error finding container aacdb4e3a9f9938b9dcbad3cb313282c82b76100196f3a1ecd81fd87d58d55eb: Status 404 returned error can't find the container with id aacdb4e3a9f9938b9dcbad3cb313282c82b76100196f3a1ecd81fd87d58d55eb
Jul 17 09:18:09.344040 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.343937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:09.347504 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.347481    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:09.349627 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.349582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zkm7q"
Jul 17 09:18:09.351471 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.351446    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea774c5b_e13d_4cd3_af7a_c46138494b3e.slice/crio-e8b2d56f9e8d4397e136c251b80d23d7cc283bbd011187a60b8951c02b936864 WatchSource:0}: Error finding container e8b2d56f9e8d4397e136c251b80d23d7cc283bbd011187a60b8951c02b936864: Status 404 returned error can't find the container with id e8b2d56f9e8d4397e136c251b80d23d7cc283bbd011187a60b8951c02b936864
Jul 17 09:18:09.354492 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.354262    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6nln8"
Jul 17 09:18:09.359701 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.359555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5wq6g"
Jul 17 09:18:09.361414 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.361393    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb36912c_324d_411b_a59b_9aa6c9bcd7b2.slice/crio-e7808f14059134f56735320dbad94fab67a134e499c275dd69d8ceab398f582f WatchSource:0}: Error finding container e7808f14059134f56735320dbad94fab67a134e499c275dd69d8ceab398f582f: Status 404 returned error can't find the container with id e7808f14059134f56735320dbad94fab67a134e499c275dd69d8ceab398f582f
Jul 17 09:18:09.363797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.363780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s2fl6"
Jul 17 09:18:09.369106 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.369081    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8684fe2_2823_4aa4_bbbd_1ebcc2239523.slice/crio-0e9a20e57ce10dc36d640853e115fa161c6647ab385e7ae2f312144d8584043c WatchSource:0}: Error finding container 0e9a20e57ce10dc36d640853e115fa161c6647ab385e7ae2f312144d8584043c: Status 404 returned error can't find the container with id 0e9a20e57ce10dc36d640853e115fa161c6647ab385e7ae2f312144d8584043c
Jul 17 09:18:09.372536 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.371861    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aeb1cf2_5fa8_448d_8aed_cfb3f875f063.slice/crio-5bfc21eb10d487085c7c4ec9a9bceb9dc2062142e92f01624d08cbf9d1f84398 WatchSource:0}: Error finding container 5bfc21eb10d487085c7c4ec9a9bceb9dc2062142e92f01624d08cbf9d1f84398: Status 404 returned error can't find the container with id 5bfc21eb10d487085c7c4ec9a9bceb9dc2062142e92f01624d08cbf9d1f84398
Jul 17 09:18:09.380563 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.380428    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da12a08_c074_4cad_b523_fffdc5fa3002.slice/crio-8f5cb1573f3adc124494d28d5e16b9891e2e1cd7260ee55d475b7c0d3bd54059 WatchSource:0}: Error finding container 8f5cb1573f3adc124494d28d5e16b9891e2e1cd7260ee55d475b7c0d3bd54059: Status 404 returned error can't find the container with id 8f5cb1573f3adc124494d28d5e16b9891e2e1cd7260ee55d475b7c0d3bd54059
Jul 17 09:18:09.384524 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:09.384500    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9bb0622_72c9_4424_a5e1_e83111a57c83.slice/crio-9eaf457b7dca04cb5ef4b4e7e25e8570897f4935a7f9cffc0d32a24a86e39da2 WatchSource:0}: Error finding container 9eaf457b7dca04cb5ef4b4e7e25e8570897f4935a7f9cffc0d32a24a86e39da2: Status 404 returned error can't find the container with id 9eaf457b7dca04cb5ef4b4e7e25e8570897f4935a7f9cffc0d32a24a86e39da2
Jul 17 09:18:09.654617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.654544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:09.654766 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.654670    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:09.654766 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.654736    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:10.654715671 +0000 UTC m=+4.046708266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:09.654766 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.654670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:09.654766 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.654753    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:09.654980 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.654805    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:10.654789548 +0000 UTC m=+4.046782149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:09.756033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:09.756004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:09.756181 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.756131    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:09.756181 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.756154    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:09.756181 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.756163    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:09.756269 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:09.756212    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:10.756194854 +0000 UTC m=+4.148187466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:10.080909 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.080735    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:13:09 +0000 UTC" deadline="2028-04-23 22:46:29.461012224 +0000 UTC"
Jul 17 09:18:10.080909 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.080804    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15517h28m19.380213231s"
Jul 17 09:18:10.147471 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.147264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:10.147471 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.147409    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:10.169824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.169785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s2fl6" event={"ID":"c9bb0622-72c9-4424-a5e1-e83111a57c83","Type":"ContainerStarted","Data":"9eaf457b7dca04cb5ef4b4e7e25e8570897f4935a7f9cffc0d32a24a86e39da2"}
Jul 17 09:18:10.174825 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.174795    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6nln8" event={"ID":"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063","Type":"ContainerStarted","Data":"5bfc21eb10d487085c7c4ec9a9bceb9dc2062142e92f01624d08cbf9d1f84398"}
Jul 17 09:18:10.181161 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.181134    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbvvs" event={"ID":"bb36912c-324d-411b-a59b-9aa6c9bcd7b2","Type":"ContainerStarted","Data":"e7808f14059134f56735320dbad94fab67a134e499c275dd69d8ceab398f582f"}
Jul 17 09:18:10.184155 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.184127    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"aacdb4e3a9f9938b9dcbad3cb313282c82b76100196f3a1ecd81fd87d58d55eb"}
Jul 17 09:18:10.192560 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.192533    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm" event={"ID":"0da71ae0-35dc-4281-b2fa-8bbea3556743","Type":"ContainerStarted","Data":"3f5122f618d1566f9a58a490f4b57501958fca985fb30e1d1d13f64e8e515f7f"}
Jul 17 09:18:10.194121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.194095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5wq6g" event={"ID":"5da12a08-c074-4cad-b523-fffdc5fa3002","Type":"ContainerStarted","Data":"8f5cb1573f3adc124494d28d5e16b9891e2e1cd7260ee55d475b7c0d3bd54059"}
Jul 17 09:18:10.196062 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.195962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zkm7q" event={"ID":"c8684fe2-2823-4aa4-bbbd-1ebcc2239523","Type":"ContainerStarted","Data":"0e9a20e57ce10dc36d640853e115fa161c6647ab385e7ae2f312144d8584043c"}
Jul 17 09:18:10.197714 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.197667    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6sldl" event={"ID":"ea774c5b-e13d-4cd3-af7a-c46138494b3e","Type":"ContainerStarted","Data":"e8b2d56f9e8d4397e136c251b80d23d7cc283bbd011187a60b8951c02b936864"}
Jul 17 09:18:10.201044 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.200997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerStarted","Data":"530144d3efa0900dc72501a09664ec8db891cfd2b2a65c06ced926f8609bd89e"}
Jul 17 09:18:10.663844 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.663806    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:10.664033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.663860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:10.664033 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.664027    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:10.664139 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.664085    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:12.664067926 +0000 UTC m=+6.056060537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:10.664217 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.664142    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:10.664217 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.664172    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:12.664162321 +0000 UTC m=+6.056154921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:10.764907 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:10.764869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:10.765091 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.765028    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:10.765091 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.765047    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:10.765091 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.765061    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:10.765264 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:10.765116    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:12.765097478 +0000 UTC m=+6.157090073 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:11.081553 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:11.081457    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:13:09 +0000 UTC" deadline="2028-03-28 15:37:03.35797068 +0000 UTC"
Jul 17 09:18:11.081553 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:11.081507    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14886h18m52.27646845s"
Jul 17 09:18:11.151268 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:11.150876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:11.151268 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:11.151011    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:11.151268 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:11.151107    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:11.151268 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:11.151201    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:12.146830 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:12.146789    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:12.147264 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.146946    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:12.684284 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:12.684205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:12.684284 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:12.684264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:12.684522 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.684475    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:12.684595 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.684544    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:16.684525409 +0000 UTC m=+10.076518012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:12.684657 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.684612    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:12.684657 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.684647    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:16.684635059 +0000 UTC m=+10.076627663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:12.786293 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:12.786258    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:12.786487 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.786445    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:12.786487 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.786466    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:12.786487 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.786479    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:12.786651 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:12.786533    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:16.786516181 +0000 UTC m=+10.178508782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:13.147491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:13.147414    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:13.147906 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:13.147548    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:13.147906 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:13.147631    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:13.147906 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:13.147716    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:14.147984 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:14.147806    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:14.147984 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:14.147931    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:15.147485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:15.147454    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:15.147641 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:15.147505    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:15.147641 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:15.147602    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:15.147773 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:15.147676    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:16.147419 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:16.147387    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:16.147900 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.147500    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:16.719196 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:16.719163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:16.719372 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:16.719212    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:16.719444 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.719360    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:16.719444 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.719377    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:16.719444 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.719434    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:24.719415642 +0000 UTC m=+18.111408239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:16.719603 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.719451    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:24.719442091 +0000 UTC m=+18.111434697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:16.819866 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:16.819824    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:16.820044 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.819974    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:16.820044 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.819988    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:16.820044 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.819996    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:16.820207 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:16.820051    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:24.820031928 +0000 UTC m=+18.212024525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:17.148153 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:17.147839    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:17.148153 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:17.147946    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:17.148153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:17.147944    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:17.148153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:17.148058    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:18.147906 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:18.147740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:18.147906 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:18.147871    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:19.146850 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:19.146816    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:19.147251 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:19.146830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:19.147251 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:19.146953    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:19.147251 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:19.146996    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:20.147723 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:20.147690    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:20.148173 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:20.147801    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:21.147719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:21.147682    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:21.147899 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:21.147813    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:21.148251 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:21.147949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:21.148251 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:21.148049    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:22.147515 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:22.147479    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:22.147704 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:22.147605    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:23.147358 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:23.147287    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:23.147821 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:23.147433    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:23.147821 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:23.147588    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:23.147821 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:23.147733    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:24.147439 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:24.147412    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:24.147801 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.147509    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:24.776399 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:24.776352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:24.776399 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:24.776401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:24.776618 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.776469    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:24.776618 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.776500    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:24.776618 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.776535    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.776515109 +0000 UTC m=+34.168507724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:24.776618 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.776555    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.776545979 +0000 UTC m=+34.168538575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:24.877403 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:24.877371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:24.877571 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.877551    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:24.877655 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.877579    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:24.877655 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.877593    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:24.877655 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:24.877650    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.877631759 +0000 UTC m=+34.269624397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:25.146997 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:25.146916    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:25.147156 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:25.147047    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:25.147156 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:25.147092    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:25.147312 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:25.147222    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:26.146983 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:26.146949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:26.147438 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:26.147077    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:27.148386 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:27.148177    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:27.148833 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:27.148248    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:27.148833 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:27.148465    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:27.148833 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:27.148564    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:28.147725 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:28.147699    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:28.147881 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:28.147810    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:29.147797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.147778    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:29.148094 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:29.147879    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:29.148094 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.147918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:29.148094 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:29.148002    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:29.239389 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.239357    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6sldl" event={"ID":"ea774c5b-e13d-4cd3-af7a-c46138494b3e","Type":"ContainerStarted","Data":"85e3e59f08d0d36b3d68334561bae1032a4a589dad572755ee007c9eeba1a4d3"}
Jul 17 09:18:29.240969 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.240943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal" event={"ID":"986b8db4c072b1a4199cd963bf9e6ab5","Type":"ContainerStarted","Data":"a02f201ecffd1181089dc21a0b7da1add2a00420d4ddecfb6d2832dfb3a95e89"}
Jul 17 09:18:29.244931 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.244908    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6nln8" event={"ID":"4aeb1cf2-5fa8-448d-8aed-cfb3f875f063","Type":"ContainerStarted","Data":"5f833f11ba4f059eb993bdc5c528fad67011c72b2f862d1d58eaf2e778ed7e66"}
Jul 17 09:18:29.246885 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.246863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"05ae24929e3196c535870fe5d2acef5f820b0198bd9b74f260fa45e5222db311"}
Jul 17 09:18:29.246953 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.246921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"3590d417fda837b2b887343ab4731a6e130351fdf70d193040b9340d61ff69e8"}
Jul 17 09:18:29.263152 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.263104    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6sldl" podStartSLOduration=2.748707053 podStartE2EDuration="22.263088829s" totalImagesPullingTime="19.514381776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:29.262923676 +0000 UTC m=+22.654916294" watchObservedRunningTime="2026-07-17 09:18:29.263088829 +0000 UTC m=+22.655081446"
Jul 17 09:18:29.314239 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:29.313703    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-140.ec2.internal" podStartSLOduration=21.313682885 podStartE2EDuration="21.313682885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:29.28584976 +0000 UTC m=+22.677842378" watchObservedRunningTime="2026-07-17 09:18:29.313682885 +0000 UTC m=+22.705675503"
Jul 17 09:18:30.147442 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.147274    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:30.147540 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:30.147509    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:30.250619 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.250590    2575 generic.go:356] "Generic (PLEG): container finished" podID="20a58c6089fe64c4052e1369fd869a24" containerID="b8650abb9c634cf61dd249b732f5445fbd13247623a61199c729e2376485c225" exitCode=0
Jul 17 09:18:30.251182 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.250664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal" event={"ID":"20a58c6089fe64c4052e1369fd869a24","Type":"ContainerDied","Data":"b8650abb9c634cf61dd249b732f5445fbd13247623a61199c729e2376485c225"}
Jul 17 09:18:30.251945 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.251917    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s2fl6" event={"ID":"c9bb0622-72c9-4424-a5e1-e83111a57c83","Type":"ContainerStarted","Data":"c8d6fb71ee44e88df9353064aa83440526316a286d729b26c01192c3121f879d"}
Jul 17 09:18:30.253252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.253229    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbvvs" event={"ID":"bb36912c-324d-411b-a59b-9aa6c9bcd7b2","Type":"ContainerStarted","Data":"7afc075c9de11e872d0ebb5cb4354b5e434adc5cb046783e2c83a1d05b17248f"}
Jul 17 09:18:30.255745 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.255727    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:18:30.256054 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256032    2575 generic.go:356] "Generic (PLEG): container finished" podID="0badef41-1489-4263-a9b7-489f7421d5c1" containerID="05ae24929e3196c535870fe5d2acef5f820b0198bd9b74f260fa45e5222db311" exitCode=1
Jul 17 09:18:30.256125 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256104    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerDied","Data":"05ae24929e3196c535870fe5d2acef5f820b0198bd9b74f260fa45e5222db311"}
Jul 17 09:18:30.256174 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256161    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"8c49df2ec169eb6f26fef1d88a9ee0d90724956db5dca7637315b4d09e195616"}
Jul 17 09:18:30.256222 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256204    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"6434f14a210fa26369cd20211913181a73a242aa0ddd838ecfb62cf06247cbe1"}
Jul 17 09:18:30.256258 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"0252831c01327f54ea5de13fdbfaae5f093faee8f0f5cd498a876589f14dee06"}
Jul 17 09:18:30.256326 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.256266    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"0f86e4eca57879cec92bf8e783de4b4e716e0385d9094eb0539bc71f279e22d7"}
Jul 17 09:18:30.257475 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.257447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm" event={"ID":"0da71ae0-35dc-4281-b2fa-8bbea3556743","Type":"ContainerStarted","Data":"553fd755c66fb08ad0586904e83a57d39d805c2331c954f482f8527ca32d2c0c"}
Jul 17 09:18:30.259750 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.259727    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5wq6g" event={"ID":"5da12a08-c074-4cad-b523-fffdc5fa3002","Type":"ContainerStarted","Data":"f613dfc282c1fefb7033ea0fa2ee7ce274d71d86a4fa83295355ab7275a9b350"}
Jul 17 09:18:30.261872 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.261849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zkm7q" event={"ID":"c8684fe2-2823-4aa4-bbbd-1ebcc2239523","Type":"ContainerStarted","Data":"c1a620fb7db423fda2c3b6448fe1e17b7d4d4ce631e829e99a9900b071675d0c"}
Jul 17 09:18:30.263266 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.263246    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="2089b4d349ac7691999e8575ca82cafef5332eea36feafe021589c070ae142ac" exitCode=0
Jul 17 09:18:30.263377 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.263338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"2089b4d349ac7691999e8575ca82cafef5332eea36feafe021589c070ae142ac"}
Jul 17 09:18:30.271427 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.271393    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6nln8" podStartSLOduration=3.729778275 podStartE2EDuration="23.271383282s" totalImagesPullingTime="19.541605007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:29.313481681 +0000 UTC m=+22.705474298" watchObservedRunningTime="2026-07-17 09:18:30.271383282 +0000 UTC m=+23.663375898"
Jul 17 09:18:30.271827 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.271807    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s2fl6" podStartSLOduration=3.89164029 podStartE2EDuration="23.271801789s" totalImagesPullingTime="19.380161499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:30.271360723 +0000 UTC m=+23.663353349" watchObservedRunningTime="2026-07-17 09:18:30.271801789 +0000 UTC m=+23.663794406"
Jul 17 09:18:30.286864 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.286706    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5wq6g" podStartSLOduration=3.854530774 podStartE2EDuration="23.286698754s" totalImagesPullingTime="19.43216798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:30.286271888 +0000 UTC m=+23.678264517" watchObservedRunningTime="2026-07-17 09:18:30.286698754 +0000 UTC m=+23.678691370"
Jul 17 09:18:30.339763 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.339730    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nbvvs" podStartSLOduration=3.8354372249999997 podStartE2EDuration="23.339719862s" totalImagesPullingTime="19.504282637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:30.339331167 +0000 UTC m=+23.731323785" watchObservedRunningTime="2026-07-17 09:18:30.339719862 +0000 UTC m=+23.731712479"
Jul 17 09:18:30.364330 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:30.364272    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zkm7q" podStartSLOduration=3.866759455 podStartE2EDuration="23.36426066s" totalImagesPullingTime="19.497501205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:30.36408805 +0000 UTC m=+23.756080667" watchObservedRunningTime="2026-07-17 09:18:30.36426066 +0000 UTC m=+23.756253277"
Jul 17 09:18:31.147244 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:31.147217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:31.147416 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:31.147356    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:31.147416 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:31.147395    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:31.147534 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:31.147486    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:31.222908 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:31.222883    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 09:18:31.267453 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:31.267386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm" event={"ID":"0da71ae0-35dc-4281-b2fa-8bbea3556743","Type":"ContainerStarted","Data":"80d129494444c30b4c9d62124ccd8346709c983e2e6539c5f65add44c210ff88"}
Jul 17 09:18:32.102368 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.102248    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T09:18:31.222904271Z","UUID":"d7e72122-2b7f-4712-85d8-03a0a10456d7","Handler":null,"Name":"","Endpoint":""}
Jul 17 09:18:32.104008 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.103984    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 09:18:32.104176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.104027    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 09:18:32.147519 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.147488    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:32.147670 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:32.147603    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:32.271076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.271040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal" event={"ID":"20a58c6089fe64c4052e1369fd869a24","Type":"ContainerStarted","Data":"cc0441765343343a91b2bb3de5bdf291a19011ed7b1443a44322d919975b48ce"}
Jul 17 09:18:32.273882 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.273862    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:18:32.274197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.274181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"e66083514597bb9d0fde5b20ed4df2c61abed99d73c53752bd7954ad6972eab0"}
Jul 17 09:18:32.333284 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.333258    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:32.333995 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.333975    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:32.365345 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:32.365225    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-140.ec2.internal" podStartSLOduration=24.365208359 podStartE2EDuration="24.365208359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:32.297886074 +0000 UTC m=+25.689878691" watchObservedRunningTime="2026-07-17 09:18:32.365208359 +0000 UTC m=+25.757200976"
Jul 17 09:18:33.147188 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.147155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:33.147188 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.147172    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:33.147541 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:33.147284    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:33.147541 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:33.147435    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:33.278664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.278623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm" event={"ID":"0da71ae0-35dc-4281-b2fa-8bbea3556743","Type":"ContainerStarted","Data":"4a27a0e6147b5f041f3f2a8c3592b6d4756408aae31ce37917ecb7fb5894b160"}
Jul 17 09:18:33.279096 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.278889    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:33.279328 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.279287    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nbvvs"
Jul 17 09:18:33.348977 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:33.348929    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m5zmm" podStartSLOduration=4.704699058 podStartE2EDuration="26.348913529s" totalImagesPullingTime="21.644214471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:33.319688594 +0000 UTC m=+26.711681211" watchObservedRunningTime="2026-07-17 09:18:33.348913529 +0000 UTC m=+26.740906146"
Jul 17 09:18:34.147932 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:34.147714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:34.148078 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:34.147996    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:34.285585 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:34.285553    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:18:34.286225 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:34.286076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"b76acea292d26f65f42f197a2587350bd9e0d21f32d239c0276857778751383a"}
Jul 17 09:18:34.286826 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:34.286801    2575 scope.go:117] "RemoveContainer" containerID="05ae24929e3196c535870fe5d2acef5f820b0198bd9b74f260fa45e5222db311"
Jul 17 09:18:35.148648 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:35.148609    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:35.148918 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:35.148754    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:35.148918 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:35.148768    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:35.148918 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:35.148852    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:36.147036 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.146776    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:36.147036 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:36.147025    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:36.155254 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.155230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tl6xw"]
Jul 17 09:18:36.155380 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.155349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:36.155465 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:36.155446    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:36.157825 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.157802    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zql5k"]
Jul 17 09:18:36.158937 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.158917    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6pxs2"]
Jul 17 09:18:36.159055 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.159022    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:36.159142 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:36.159123    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:36.291815 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.291782    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="d33ce62405f509e4a70911957cc6d7d7b52f9dfa10f7321bd2bd7f3e2c6c37f0" exitCode=0
Jul 17 09:18:36.291989 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.291865    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"d33ce62405f509e4a70911957cc6d7d7b52f9dfa10f7321bd2bd7f3e2c6c37f0"}
Jul 17 09:18:36.295437 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.295417    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:18:36.295869 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.295840    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:36.295958 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.295869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" event={"ID":"0badef41-1489-4263-a9b7-489f7421d5c1","Type":"ContainerStarted","Data":"43f1525f207a53bffb5e0199dde721ecf3cd0e142e1d57165f7396aae7abac08"}
Jul 17 09:18:36.296011 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:36.295948    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:36.296186 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.296151    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:36.296273 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.296217    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:36.296273 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.296264    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:36.311655 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.311628    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:36.312060 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.312040    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:18:36.352749 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:36.352708    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" podStartSLOduration=9.768766922 podStartE2EDuration="29.352694478s" totalImagesPullingTime="19.583927556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:36.352486817 +0000 UTC m=+29.744479434" watchObservedRunningTime="2026-07-17 09:18:36.352694478 +0000 UTC m=+29.744687073"
Jul 17 09:18:38.147220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.147187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:38.147220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.147201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:38.147220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.147203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:38.147804 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:38.147321    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:38.147804 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:38.147368    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:38.147804 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:38.147456    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:38.302608 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.302582    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="e46740b4f2a60f7526154c0742af5818d9ceceacd2d4bbabf8a375deb159905a" exitCode=0
Jul 17 09:18:38.302746 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.302664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"e46740b4f2a60f7526154c0742af5818d9ceceacd2d4bbabf8a375deb159905a"}
Jul 17 09:18:38.320935 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:38.320890    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg" podUID="0badef41-1489-4263-a9b7-489f7421d5c1" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 17 09:18:40.147814 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.147653    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:40.148280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.147653    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:40.148280 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.147904    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tl6xw" podUID="7849ad77-4297-4d63-97eb-4f60039e4a1b"
Jul 17 09:18:40.148280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.147653    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:40.148280 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.148024    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6pxs2" podUID="cc5779d7-9440-4f6f-b8e1-4ca12489f555"
Jul 17 09:18:40.148280 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.147954    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zql5k" podUID="aed431b3-68e9-4fb4-9ab3-bdd3322f7c10"
Jul 17 09:18:40.308786 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.308714    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="935a1bf95765741d3944681ecca0cdd86cc8aaa3395230de28614644bc4b81e5" exitCode=0
Jul 17 09:18:40.308913 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.308785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"935a1bf95765741d3944681ecca0cdd86cc8aaa3395230de28614644bc4b81e5"}
Jul 17 09:18:40.803251 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.803228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:40.803408 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.803260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:40.803408 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.803381    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:40.803486 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.803432    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret podName:7849ad77-4297-4d63-97eb-4f60039e4a1b nodeName:}" failed. No retries permitted until 2026-07-17 09:19:12.803418877 +0000 UTC m=+66.195411472 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret") pod "global-pull-secret-syncer-tl6xw" (UID: "7849ad77-4297-4d63-97eb-4f60039e4a1b") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:40.803486 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.803382    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:40.803557 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.803503    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:12.803490322 +0000 UTC m=+66.195482921 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:40.904572 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:40.904536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:40.904732 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.904710    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:40.904780 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.904740    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:40.904780 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.904754    2575 projected.go:196] Error preparing data for projected volume kube-api-access-bkjdh for pod openshift-network-diagnostics/network-check-target-zql5k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:40.904861 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:40.904810    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh podName:aed431b3-68e9-4fb4-9ab3-bdd3322f7c10 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:12.904797424 +0000 UTC m=+66.296790020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkjdh" (UniqueName: "kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh") pod "network-check-target-zql5k" (UID: "aed431b3-68e9-4fb4-9ab3-bdd3322f7c10") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:41.930346 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.930265    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-140.ec2.internal" event="NodeReady"
Jul 17 09:18:41.930691 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.930400    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 09:18:41.980977 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.980951    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4jtlg"]
Jul 17 09:18:41.994360 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.994337    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-46ht7"]
Jul 17 09:18:41.994485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.994450    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:41.997071 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.997052    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 09:18:41.997770 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.997556    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4mh5z\""
Jul 17 09:18:41.997873 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:41.997849    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 09:18:42.009020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.008985    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-46ht7"]
Jul 17 09:18:42.009020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.009023    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4jtlg"]
Jul 17 09:18:42.009177 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.009111    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.011798 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.011778    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 09:18:42.012617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.011982    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 09:18:42.012617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.011842    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 09:18:42.012617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.012212    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cdjx7\""
Jul 17 09:18:42.113331 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113289    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m55hj\" (UniqueName: \"kubernetes.io/projected/cc8a050f-5f64-4c67-b0a6-93d5877df02d-kube-api-access-m55hj\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.113331 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc8a050f-5f64-4c67-b0a6-93d5877df02d-config-volume\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.113570 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113468    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkxgv\" (UniqueName: \"kubernetes.io/projected/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-kube-api-access-kkxgv\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.113570 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-cert\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.113570 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113558    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc8a050f-5f64-4c67-b0a6-93d5877df02d-tmp-dir\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.113697 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.113577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.147513 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.147487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:18:42.147639 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.147487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:18:42.147687 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.147487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:18:42.153473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.153453    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:18:42.154458 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.154402    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:18:42.154532 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.154515    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:18:42.154654 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.154634    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzpfx\""
Jul 17 09:18:42.154762 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.154634    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2lf7\""
Jul 17 09:18:42.155047 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.155030    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:18:42.214200 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214153    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxgv\" (UniqueName: \"kubernetes.io/projected/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-kube-api-access-kkxgv\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.214200 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-cert\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.214353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc8a050f-5f64-4c67-b0a6-93d5877df02d-tmp-dir\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.214353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.214353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m55hj\" (UniqueName: \"kubernetes.io/projected/cc8a050f-5f64-4c67-b0a6-93d5877df02d-kube-api-access-m55hj\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.214353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc8a050f-5f64-4c67-b0a6-93d5877df02d-config-volume\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.214536 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:42.214411    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:42.214536 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:42.214479    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:18:42.714458187 +0000 UTC m=+36.106450805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:18:42.214536 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214502    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc8a050f-5f64-4c67-b0a6-93d5877df02d-tmp-dir\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.214793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.214775    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cc8a050f-5f64-4c67-b0a6-93d5877df02d-config-volume\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.224050 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.224029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m55hj\" (UniqueName: \"kubernetes.io/projected/cc8a050f-5f64-4c67-b0a6-93d5877df02d-kube-api-access-m55hj\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.224587 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.224563    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkxgv\" (UniqueName: \"kubernetes.io/projected/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-kube-api-access-kkxgv\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.227947 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.227928    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/234e65f8-8a04-40a6-bbcc-4f45ea67d6c4-cert\") pod \"ingress-canary-46ht7\" (UID: \"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4\") " pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.319361 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.319336    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-46ht7"
Jul 17 09:18:42.464771 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.464707    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-46ht7"]
Jul 17 09:18:42.468533 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:18:42.468509    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234e65f8_8a04_40a6_bbcc_4f45ea67d6c4.slice/crio-7fd56639c895e307617fa502f1170d9d33989fe810ccadba3e65085531794bd2 WatchSource:0}: Error finding container 7fd56639c895e307617fa502f1170d9d33989fe810ccadba3e65085531794bd2: Status 404 returned error can't find the container with id 7fd56639c895e307617fa502f1170d9d33989fe810ccadba3e65085531794bd2
Jul 17 09:18:42.717935 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:42.717853    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:42.718094 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:42.718021    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:42.718148 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:42.718107    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:18:43.718085683 +0000 UTC m=+37.110078290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:18:43.318045 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:43.318006    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-46ht7" event={"ID":"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4","Type":"ContainerStarted","Data":"7fd56639c895e307617fa502f1170d9d33989fe810ccadba3e65085531794bd2"}
Jul 17 09:18:43.726667 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:43.726629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:43.726848 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:43.726800    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:43.726914 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:43.726883    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:18:45.726859377 +0000 UTC m=+39.118851992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:18:45.741716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:45.741506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:45.742168 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:45.741650    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:45.742168 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:45.741836    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:18:49.741811401 +0000 UTC m=+43.133804005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:18:47.327539 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:47.327498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-46ht7" event={"ID":"234e65f8-8a04-40a6-bbcc-4f45ea67d6c4","Type":"ContainerStarted","Data":"c3b22ee0df30bfeef554f64cb39ad53b718461e04ee92581f317f07cbd05d4dc"}
Jul 17 09:18:47.332427 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:47.332400    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="2d220cd4735f1dc4d516aca8d588957ac23d1b41460ffcd043742b54000a2032" exitCode=0
Jul 17 09:18:47.332585 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:47.332435    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"2d220cd4735f1dc4d516aca8d588957ac23d1b41460ffcd043742b54000a2032"}
Jul 17 09:18:47.345495 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:47.345451    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-46ht7" podStartSLOduration=2.518963355 podStartE2EDuration="6.345436865s" totalImagesPullingTime="3.82647351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:47.344730821 +0000 UTC m=+40.736723438" watchObservedRunningTime="2026-07-17 09:18:47.345436865 +0000 UTC m=+40.737429480"
Jul 17 09:18:48.337863 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:48.337830    2575 generic.go:356] "Generic (PLEG): container finished" podID="8b42ab0d-e694-4cf6-9054-819cf82b1790" containerID="1195e38ed3d7f7023821767bca22b237da6496b3372e60be8328a7f434cf369e" exitCode=0
Jul 17 09:18:48.338273 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:48.337908    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerDied","Data":"1195e38ed3d7f7023821767bca22b237da6496b3372e60be8328a7f434cf369e"}
Jul 17 09:18:49.343726 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:49.343684    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" event={"ID":"8b42ab0d-e694-4cf6-9054-819cf82b1790","Type":"ContainerStarted","Data":"f0dc334a48fb455c2930370fea9e388eac34afcdc52bd8fe6af8ffae1efc8928"}
Jul 17 09:18:49.385352 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:49.385293    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ssbbs" podStartSLOduration=9.341338251 podStartE2EDuration="42.385278117s" totalImagesPullingTime="33.043939866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:49.383596168 +0000 UTC m=+42.775588799" watchObservedRunningTime="2026-07-17 09:18:49.385278117 +0000 UTC m=+42.777270733"
Jul 17 09:18:49.766892 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:49.766864    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:49.767034 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:49.767004    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:49.767073 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:49.767062    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:18:57.767047988 +0000 UTC m=+51.159040583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:18:57.819950 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:18:57.819906    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:18:57.820451 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:57.820073    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:18:57.820451 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:18:57.820141    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:19:13.820124538 +0000 UTC m=+67.212117133 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:08.311825 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:08.311799    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lhrpg"
Jul 17 09:19:12.822507 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.822471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:19:12.822507 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.822511    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:19:12.825393 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.825363    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:19:12.825466 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.825456    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:19:12.833581 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:12.833543    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 09:19:12.833646 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:12.833618    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs podName:cc5779d7-9440-4f6f-b8e1-4ca12489f555 nodeName:}" failed. No retries permitted until 2026-07-17 09:20:16.833602856 +0000 UTC m=+130.225595455 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs") pod "network-metrics-daemon-6pxs2" (UID: "cc5779d7-9440-4f6f-b8e1-4ca12489f555") : secret "metrics-daemon-secret" not found
Jul 17 09:19:12.836573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.836546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7849ad77-4297-4d63-97eb-4f60039e4a1b-original-pull-secret\") pod \"global-pull-secret-syncer-tl6xw\" (UID: \"7849ad77-4297-4d63-97eb-4f60039e4a1b\") " pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:19:12.922966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.922931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:19:12.926377 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.926359    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:19:12.947534 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.947516    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:19:12.956293 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:12.956265    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjdh\" (UniqueName: \"kubernetes.io/projected/aed431b3-68e9-4fb4-9ab3-bdd3322f7c10-kube-api-access-bkjdh\") pod \"network-check-target-zql5k\" (UID: \"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10\") " pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:19:13.060460 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.060440    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzpfx\""
Jul 17 09:19:13.067848 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.067830    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:19:13.067909 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.067855    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tl6xw"
Jul 17 09:19:13.196532 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.196503    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zql5k"]
Jul 17 09:19:13.199534 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:13.199512    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed431b3_68e9_4fb4_9ab3_bdd3322f7c10.slice/crio-af2e8ffb4c157d706ab62decbf4bda5c4767189b90946e058dfbe6259255b387 WatchSource:0}: Error finding container af2e8ffb4c157d706ab62decbf4bda5c4767189b90946e058dfbe6259255b387: Status 404 returned error can't find the container with id af2e8ffb4c157d706ab62decbf4bda5c4767189b90946e058dfbe6259255b387
Jul 17 09:19:13.209572 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.209546    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tl6xw"]
Jul 17 09:19:13.213112 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:13.213088    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7849ad77_4297_4d63_97eb_4f60039e4a1b.slice/crio-774f6e4686c5fc9b89e08cf3a17866b980f0fc3103a96f70e63cc962adb7b365 WatchSource:0}: Error finding container 774f6e4686c5fc9b89e08cf3a17866b980f0fc3103a96f70e63cc962adb7b365: Status 404 returned error can't find the container with id 774f6e4686c5fc9b89e08cf3a17866b980f0fc3103a96f70e63cc962adb7b365
Jul 17 09:19:13.401362 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.401271    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tl6xw" event={"ID":"7849ad77-4297-4d63-97eb-4f60039e4a1b","Type":"ContainerStarted","Data":"774f6e4686c5fc9b89e08cf3a17866b980f0fc3103a96f70e63cc962adb7b365"}
Jul 17 09:19:13.402247 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.402225    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zql5k" event={"ID":"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10","Type":"ContainerStarted","Data":"af2e8ffb4c157d706ab62decbf4bda5c4767189b90946e058dfbe6259255b387"}
Jul 17 09:19:13.828949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:13.828913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:19:13.829496 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:13.829080    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:13.829496 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:13.829165    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls podName:cc8a050f-5f64-4c67-b0a6-93d5877df02d nodeName:}" failed. No retries permitted until 2026-07-17 09:19:45.829140845 +0000 UTC m=+99.221133451 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls") pod "dns-default-4jtlg" (UID: "cc8a050f-5f64-4c67-b0a6-93d5877df02d") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:18.417611 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:18.417569    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tl6xw" event={"ID":"7849ad77-4297-4d63-97eb-4f60039e4a1b","Type":"ContainerStarted","Data":"18bb86cae8329fc4d8728ac77d04ab9d4060858eeef828709775c99567af5ba9"}
Jul 17 09:19:18.419001 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:18.418971    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zql5k" event={"ID":"aed431b3-68e9-4fb4-9ab3-bdd3322f7c10","Type":"ContainerStarted","Data":"88d2134b334e091d228d0ab295ea23df63daf96c6a73371edb34dd2db245132b"}
Jul 17 09:19:18.419131 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:18.419020    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:19:18.438093 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:18.438046    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tl6xw" podStartSLOduration=67.00646059 podStartE2EDuration="1m11.438030691s" totalImagesPullingTime="4.431570101s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:18.437773841 +0000 UTC m=+71.829766466" watchObservedRunningTime="2026-07-17 09:19:18.438030691 +0000 UTC m=+71.830023308"
Jul 17 09:19:18.457516 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:18.457474    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zql5k" podStartSLOduration=67.017347867 podStartE2EDuration="1m11.457460305s" totalImagesPullingTime="4.440112438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:18.456828359 +0000 UTC m=+71.848820977" watchObservedRunningTime="2026-07-17 09:19:18.457460305 +0000 UTC m=+71.849452899"
Jul 17 09:19:30.758780 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.758676    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"]
Jul 17 09:19:30.762530 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.762515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.766098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.766072    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 17 09:19:30.766098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.766091    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xxjhh\""
Jul 17 09:19:30.767476 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.767454    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.767582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.767460    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.767582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.767464    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 17 09:19:30.776333 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.776313    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"]
Jul 17 09:19:30.835877 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.835854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ff6e71b-3867-44d1-a167-4625c3d3cd25-config\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.836005 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.835892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mh8c\" (UniqueName: \"kubernetes.io/projected/2ff6e71b-3867-44d1-a167-4625c3d3cd25-kube-api-access-9mh8c\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.836005 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.835929    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ff6e71b-3867-44d1-a167-4625c3d3cd25-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.887310 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.887271    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"]
Jul 17 09:19:30.890169 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.890153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:30.900674 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.900657    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 17 09:19:30.900788 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.900768    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.900853 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.900838    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 17 09:19:30.902025 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.902002    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xs7cp\""
Jul 17 09:19:30.902097 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.902065    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.922088 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.922065    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rpjth"]
Jul 17 09:19:30.924819 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.924800    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"]
Jul 17 09:19:30.924923 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.924876    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:30.927512 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.927492    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rrdgm\""
Jul 17 09:19:30.927609 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.927534    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 09:19:30.929994 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.929972    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.930091 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.930069    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 09:19:30.934143 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.934127    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.936186 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936168    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fjgs\" (UniqueName: \"kubernetes.io/projected/a9d6b919-6402-4646-ab7f-0dfe1e440e49-kube-api-access-5fjgs\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:30.936264 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936199    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9d6b919-6402-4646-ab7f-0dfe1e440e49-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:30.936264 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936238    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ff6e71b-3867-44d1-a167-4625c3d3cd25-config\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.936391 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9d6b919-6402-4646-ab7f-0dfe1e440e49-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:30.936391 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936330    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mh8c\" (UniqueName: \"kubernetes.io/projected/2ff6e71b-3867-44d1-a167-4625c3d3cd25-kube-api-access-9mh8c\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.936391 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ff6e71b-3867-44d1-a167-4625c3d3cd25-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.936742 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.936723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2ff6e71b-3867-44d1-a167-4625c3d3cd25-config\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.938457 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.938440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2ff6e71b-3867-44d1-a167-4625c3d3cd25-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.949238 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.949219    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 09:19:30.971610 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.971586    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rpjth"]
Jul 17 09:19:30.977707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.977689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mh8c\" (UniqueName: \"kubernetes.io/projected/2ff6e71b-3867-44d1-a167-4625c3d3cd25-kube-api-access-9mh8c\") pod \"service-ca-operator-5bcbc94c78-jxhk6\" (UID: \"2ff6e71b-3867-44d1-a167-4625c3d3cd25\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:30.995489 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.995466    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"]
Jul 17 09:19:30.998387 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:30.998373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.001603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.001588    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 17 09:19:31.001715 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.001696    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 17 09:19:31.001795 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.001774    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hm2vp\""
Jul 17 09:19:31.001980 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.001965    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:31.018412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.018365    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"]
Jul 17 09:19:31.036972 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.036948    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qwk2\" (UniqueName: \"kubernetes.io/projected/8e31e400-40be-4561-860f-7c83a2fa2da3-kube-api-access-8qwk2\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.037061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.036977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9d6b919-6402-4646-ab7f-0dfe1e440e49-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.037061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.036995    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-config\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.037061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3528b82-4841-475a-9e5b-5d94d683ac5c-serving-cert\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.037061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037027    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr8nj\" (UniqueName: \"kubernetes.io/projected/b3528b82-4841-475a-9e5b-5d94d683ac5c-kube-api-access-jr8nj\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.037245 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037090    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-trusted-ca\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.037245 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037165    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.037245 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fjgs\" (UniqueName: \"kubernetes.io/projected/a9d6b919-6402-4646-ab7f-0dfe1e440e49-kube-api-access-5fjgs\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.037393 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037263    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9d6b919-6402-4646-ab7f-0dfe1e440e49-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.037492 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.037472    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9d6b919-6402-4646-ab7f-0dfe1e440e49-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.039155 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.039137    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9d6b919-6402-4646-ab7f-0dfe1e440e49-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.052327 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.052290    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fjgs\" (UniqueName: \"kubernetes.io/projected/a9d6b919-6402-4646-ab7f-0dfe1e440e49-kube-api-access-5fjgs\") pod \"kube-storage-version-migrator-operator-7658d9b749-759tr\" (UID: \"a9d6b919-6402-4646-ab7f-0dfe1e440e49\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.071366 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.071346    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"
Jul 17 09:19:31.137850 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.137825    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-trusted-ca\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.137975 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.137871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.137975 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:31.137966    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:19:31.138041 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:31.138022    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls podName:8e31e400-40be-4561-860f-7c83a2fa2da3 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:31.638001786 +0000 UTC m=+85.029994381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsxvz" (UID: "8e31e400-40be-4561-860f-7c83a2fa2da3") : secret "samples-operator-tls" not found
Jul 17 09:19:31.138688 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.138259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qwk2\" (UniqueName: \"kubernetes.io/projected/8e31e400-40be-4561-860f-7c83a2fa2da3-kube-api-access-8qwk2\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.138688 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.138289    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-config\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.138688 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.138342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3528b82-4841-475a-9e5b-5d94d683ac5c-serving-cert\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.138688 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.138362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr8nj\" (UniqueName: \"kubernetes.io/projected/b3528b82-4841-475a-9e5b-5d94d683ac5c-kube-api-access-jr8nj\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.138870 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.138856    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-trusted-ca\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.139069 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.139042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b3528b82-4841-475a-9e5b-5d94d683ac5c-config\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.140456 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.140423    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b3528b82-4841-475a-9e5b-5d94d683ac5c-serving-cert\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.163701 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.163678    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qwk2\" (UniqueName: \"kubernetes.io/projected/8e31e400-40be-4561-860f-7c83a2fa2da3-kube-api-access-8qwk2\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.163847 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.163822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr8nj\" (UniqueName: \"kubernetes.io/projected/b3528b82-4841-475a-9e5b-5d94d683ac5c-kube-api-access-jr8nj\") pod \"console-operator-565b7b569d-rpjth\" (UID: \"b3528b82-4841-475a-9e5b-5d94d683ac5c\") " pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.197795 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.197769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"
Jul 17 09:19:31.198879 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.198858    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6"]
Jul 17 09:19:31.201971 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:31.201948    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff6e71b_3867_44d1_a167_4625c3d3cd25.slice/crio-b493abdcfe9df2388986048880fc0caf38b50120026258f5995fe3acf63adbcb WatchSource:0}: Error finding container b493abdcfe9df2388986048880fc0caf38b50120026258f5995fe3acf63adbcb: Status 404 returned error can't find the container with id b493abdcfe9df2388986048880fc0caf38b50120026258f5995fe3acf63adbcb
Jul 17 09:19:31.232846 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.232820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:31.326689 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.326661    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr"]
Jul 17 09:19:31.330053 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:31.330026    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d6b919_6402_4646_ab7f_0dfe1e440e49.slice/crio-7c39773e1e8da0b558e30c1e577f2112aa21be9a7b63efce1cad3976ddeb4a24 WatchSource:0}: Error finding container 7c39773e1e8da0b558e30c1e577f2112aa21be9a7b63efce1cad3976ddeb4a24: Status 404 returned error can't find the container with id 7c39773e1e8da0b558e30c1e577f2112aa21be9a7b63efce1cad3976ddeb4a24
Jul 17 09:19:31.371240 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.371214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-rpjth"]
Jul 17 09:19:31.373735 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:31.373709    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3528b82_4841_475a_9e5b_5d94d683ac5c.slice/crio-7138a0c0ac9084f39c04a5cc18157067a6d4193a187fdbe140732a98dd0f0357 WatchSource:0}: Error finding container 7138a0c0ac9084f39c04a5cc18157067a6d4193a187fdbe140732a98dd0f0357: Status 404 returned error can't find the container with id 7138a0c0ac9084f39c04a5cc18157067a6d4193a187fdbe140732a98dd0f0357
Jul 17 09:19:31.449353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.449324    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr" event={"ID":"a9d6b919-6402-4646-ab7f-0dfe1e440e49","Type":"ContainerStarted","Data":"7c39773e1e8da0b558e30c1e577f2112aa21be9a7b63efce1cad3976ddeb4a24"}
Jul 17 09:19:31.450338 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.450318    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6" event={"ID":"2ff6e71b-3867-44d1-a167-4625c3d3cd25","Type":"ContainerStarted","Data":"b493abdcfe9df2388986048880fc0caf38b50120026258f5995fe3acf63adbcb"}
Jul 17 09:19:31.451370 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.451350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" event={"ID":"b3528b82-4841-475a-9e5b-5d94d683ac5c","Type":"ContainerStarted","Data":"7138a0c0ac9084f39c04a5cc18157067a6d4193a187fdbe140732a98dd0f0357"}
Jul 17 09:19:31.642259 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:31.642175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:31.642408 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:31.642342    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:19:31.642408 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:31.642406    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls podName:8e31e400-40be-4561-860f-7c83a2fa2da3 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:32.642391901 +0000 UTC m=+86.034384496 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsxvz" (UID: "8e31e400-40be-4561-860f-7c83a2fa2da3") : secret "samples-operator-tls" not found
Jul 17 09:19:32.650491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:32.650454    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:32.650965 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:32.650597    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:19:32.650965 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:32.650670    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls podName:8e31e400-40be-4561-860f-7c83a2fa2da3 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.650650073 +0000 UTC m=+88.042642671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsxvz" (UID: "8e31e400-40be-4561-860f-7c83a2fa2da3") : secret "samples-operator-tls" not found
Jul 17 09:19:34.313148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.313126    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ls2cv"]
Jul 17 09:19:34.316433 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.316414    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.322627 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.322608    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 09:19:34.323124 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.323100    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cb5vz\""
Jul 17 09:19:34.323846 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.323824    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 09:19:34.323846 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.323839    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 09:19:34.324176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.324156    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 09:19:34.347982 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.347961    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ls2cv"]
Jul 17 09:19:34.361614 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.361586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.361707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.361620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.361707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.361657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjb89\" (UniqueName: \"kubernetes.io/projected/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-api-access-pjb89\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.361707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.361680    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-data-volume\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.361818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.361739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-crio-socket\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462153 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.462139    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.462270 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462167    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjb89\" (UniqueName: \"kubernetes.io/projected/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-api-access-pjb89\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462270 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.462191    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.462270 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.462203    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.962181572 +0000 UTC m=+88.354174166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.462270 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462227    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-data-volume\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462270 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.462255    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.962237169 +0000 UTC m=+88.354229782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.462543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-crio-socket\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462458    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-crio-socket\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.462682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.462661    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-data-volume\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.463026 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.463007    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/0.log"
Jul 17 09:19:34.463103 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.463046    2575 generic.go:356] "Generic (PLEG): container finished" podID="b3528b82-4841-475a-9e5b-5d94d683ac5c" containerID="2adf7b54456d2b2d10149c4e6dbec29384844d177ed68865d8c135dba0ddea58" exitCode=255
Jul 17 09:19:34.463160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.463130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" event={"ID":"b3528b82-4841-475a-9e5b-5d94d683ac5c","Type":"ContainerDied","Data":"2adf7b54456d2b2d10149c4e6dbec29384844d177ed68865d8c135dba0ddea58"}
Jul 17 09:19:34.463395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.463376    2575 scope.go:117] "RemoveContainer" containerID="2adf7b54456d2b2d10149c4e6dbec29384844d177ed68865d8c135dba0ddea58"
Jul 17 09:19:34.464818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.464797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr" event={"ID":"a9d6b919-6402-4646-ab7f-0dfe1e440e49","Type":"ContainerStarted","Data":"c25779bc7aa4ec4655a14eb036b637693c6b70c592d865ebf5e6f7c393d52611"}
Jul 17 09:19:34.466395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.466371    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6" event={"ID":"2ff6e71b-3867-44d1-a167-4625c3d3cd25","Type":"ContainerStarted","Data":"1553a6fe269b69726e2d4a3449128e52a0af15853d7aa5bdd6ca646549630687"}
Jul 17 09:19:34.476860 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.476839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjb89\" (UniqueName: \"kubernetes.io/projected/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-api-access-pjb89\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.548901 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.548851    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6" podStartSLOduration=1.441352982 podStartE2EDuration="4.548833083s" totalImagesPullingTime="3.107480101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:34.5484968 +0000 UTC m=+87.940489435" watchObservedRunningTime="2026-07-17 09:19:34.548833083 +0000 UTC m=+87.940825701"
Jul 17 09:19:34.549548 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.549507    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr" podStartSLOduration=1.5756275579999999 podStartE2EDuration="4.549486005s" totalImagesPullingTime="2.973858447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:34.509713936 +0000 UTC m=+87.901706553" watchObservedRunningTime="2026-07-17 09:19:34.549486005 +0000 UTC m=+87.941478628"
Jul 17 09:19:34.663636 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.663601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:34.663772 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.663738    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:19:34.663827 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.663799    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls podName:8e31e400-40be-4561-860f-7c83a2fa2da3 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:38.663783606 +0000 UTC m=+92.055776201 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsxvz" (UID: "8e31e400-40be-4561-860f-7c83a2fa2da3") : secret "samples-operator-tls" not found
Jul 17 09:19:34.965942 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.965857    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.965942 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:34.965895    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:34.966153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.965999    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.966153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.966021    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.966153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.966048    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:35.966035232 +0000 UTC m=+89.358027827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.966153 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:34.966087    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:35.966068954 +0000 UTC m=+89.358061580 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:35.355864 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.355835    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"]
Jul 17 09:19:35.358906 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.358876    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"]
Jul 17 09:19:35.359044 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.358906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.361775 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.361755    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.361884 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.361811    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 09:19:35.362775 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.362760    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 09:19:35.362846 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.362762    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 09:19:35.362846 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.362820    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 09:19:35.366531 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.366510    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fq82v\""
Jul 17 09:19:35.366637 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.366619    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 17 09:19:35.374752 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.374736    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"]
Jul 17 09:19:35.375642 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.375622    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"]
Jul 17 09:19:35.468968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.468936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkqr\" (UniqueName: \"kubernetes.io/projected/f6f75afb-1236-4caf-9f51-a8dc38604476-kube-api-access-7dkqr\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.469107 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.468987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6f75afb-1236-4caf-9f51-a8dc38604476-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.469107 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.469055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e33c939f-2a5b-4186-9179-e558f477815c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.469210 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.469119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f75afb-1236-4caf-9f51-a8dc38604476-tmp\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.469210 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.469143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kblwv\" (UniqueName: \"kubernetes.io/projected/e33c939f-2a5b-4186-9179-e558f477815c-kube-api-access-kblwv\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.471275 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.471254    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:19:35.471816 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.471798    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/0.log"
Jul 17 09:19:35.471896 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.471838    2575 generic.go:356] "Generic (PLEG): container finished" podID="b3528b82-4841-475a-9e5b-5d94d683ac5c" containerID="89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1" exitCode=255
Jul 17 09:19:35.471955 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.471918    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" event={"ID":"b3528b82-4841-475a-9e5b-5d94d683ac5c","Type":"ContainerDied","Data":"89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1"}
Jul 17 09:19:35.472018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.471977    2575 scope.go:117] "RemoveContainer" containerID="2adf7b54456d2b2d10149c4e6dbec29384844d177ed68865d8c135dba0ddea58"
Jul 17 09:19:35.472175 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.472150    2575 scope.go:117] "RemoveContainer" containerID="89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1"
Jul 17 09:19:35.472378 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:35.472355    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rpjth_openshift-console-operator(b3528b82-4841-475a-9e5b-5d94d683ac5c)\"" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" podUID="b3528b82-4841-475a-9e5b-5d94d683ac5c"
Jul 17 09:19:35.570146 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570115    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e33c939f-2a5b-4186-9179-e558f477815c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.570265 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f75afb-1236-4caf-9f51-a8dc38604476-tmp\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.570452 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kblwv\" (UniqueName: \"kubernetes.io/projected/e33c939f-2a5b-4186-9179-e558f477815c-kube-api-access-kblwv\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.570561 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkqr\" (UniqueName: \"kubernetes.io/projected/f6f75afb-1236-4caf-9f51-a8dc38604476-kube-api-access-7dkqr\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.570620 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6f75afb-1236-4caf-9f51-a8dc38604476-tmp\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.570820 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.570800    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6f75afb-1236-4caf-9f51-a8dc38604476-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.572759 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.572738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e33c939f-2a5b-4186-9179-e558f477815c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.573431 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.573389    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f6f75afb-1236-4caf-9f51-a8dc38604476-klusterlet-config\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.579559 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.579538    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kblwv\" (UniqueName: \"kubernetes.io/projected/e33c939f-2a5b-4186-9179-e558f477815c-kube-api-access-kblwv\") pod \"managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb\" (UID: \"e33c939f-2a5b-4186-9179-e558f477815c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.580313 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.580274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkqr\" (UniqueName: \"kubernetes.io/projected/f6f75afb-1236-4caf-9f51-a8dc38604476-kube-api-access-7dkqr\") pod \"klusterlet-addon-workmgr-5fc44677d7-b7v2j\" (UID: \"f6f75afb-1236-4caf-9f51-a8dc38604476\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.671069 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.671037    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:35.681180 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.681077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"
Jul 17 09:19:35.948863 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:35.948833    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f75afb_1236_4caf_9f51_a8dc38604476.slice/crio-7ba3fdb6a6a23d25562a80c06b9ded30008467e262192767e6142371a1507ee0 WatchSource:0}: Error finding container 7ba3fdb6a6a23d25562a80c06b9ded30008467e262192767e6142371a1507ee0: Status 404 returned error can't find the container with id 7ba3fdb6a6a23d25562a80c06b9ded30008467e262192767e6142371a1507ee0
Jul 17 09:19:35.973266 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.973237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:35.973377 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.973287    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:35.973419 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:35.973389    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:35.973452 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:35.973433    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:37.973419462 +0000 UTC m=+91.365412057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:35.973452 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:35.973388    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:35.973531 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:35.973465    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:37.97345792 +0000 UTC m=+91.365450515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:35.980673 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:35.980652    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode33c939f_2a5b_4186_9179_e558f477815c.slice/crio-b0dd7cd95449b45af6f4eb0a82e94c816a50ee5188044574356c23a83301c4d9 WatchSource:0}: Error finding container b0dd7cd95449b45af6f4eb0a82e94c816a50ee5188044574356c23a83301c4d9: Status 404 returned error can't find the container with id b0dd7cd95449b45af6f4eb0a82e94c816a50ee5188044574356c23a83301c4d9
Jul 17 09:19:35.982759 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.982735    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"]
Jul 17 09:19:35.983690 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:35.983672    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb"]
Jul 17 09:19:36.464688 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:36.464656    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zkm7q_c8684fe2-2823-4aa4-bbbd-1ebcc2239523/dns-node-resolver/0.log"
Jul 17 09:19:36.478226 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:36.478199    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:19:36.478617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:36.478600    2575 scope.go:117] "RemoveContainer" containerID="89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1"
Jul 17 09:19:36.478847 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:36.478819    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rpjth_openshift-console-operator(b3528b82-4841-475a-9e5b-5d94d683ac5c)\"" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" podUID="b3528b82-4841-475a-9e5b-5d94d683ac5c"
Jul 17 09:19:36.479898 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:36.479853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb" event={"ID":"e33c939f-2a5b-4186-9179-e558f477815c","Type":"ContainerStarted","Data":"b0dd7cd95449b45af6f4eb0a82e94c816a50ee5188044574356c23a83301c4d9"}
Jul 17 09:19:36.481325 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:36.481290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j" event={"ID":"f6f75afb-1236-4caf-9f51-a8dc38604476","Type":"ContainerStarted","Data":"7ba3fdb6a6a23d25562a80c06b9ded30008467e262192767e6142371a1507ee0"}
Jul 17 09:19:37.060009 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:37.059981    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5wq6g_5da12a08-c074-4cad-b523-fffdc5fa3002/node-ca/0.log"
Jul 17 09:19:37.870421 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:37.870392    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-46ht7_234e65f8-8a04-40a6-bbcc-4f45ea67d6c4/serve-healthcheck-canary/0.log"
Jul 17 09:19:37.989337 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:37.989153    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:37.989337 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:37.989271    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:37.989337 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:37.989318    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:37.989596 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:37.989408    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:37.989596 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:37.989411    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:41.989387616 +0000 UTC m=+95.381380225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:37.989596 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:37.989470    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:41.989455124 +0000 UTC m=+95.381447721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:38.695353 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:38.695322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:38.695538 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:38.695493    2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:19:38.695595 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:38.695560    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls podName:8e31e400-40be-4561-860f-7c83a2fa2da3 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:46.695539289 +0000 UTC m=+100.087531885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-dsxvz" (UID: "8e31e400-40be-4561-860f-7c83a2fa2da3") : secret "samples-operator-tls" not found
Jul 17 09:19:40.496409 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.496369    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb" event={"ID":"e33c939f-2a5b-4186-9179-e558f477815c","Type":"ContainerStarted","Data":"86d6e3c9b258137b8483c81675724705eace2ebcc4fb7d8d69c0413af766dd98"}
Jul 17 09:19:40.497794 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.497762    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j" event={"ID":"f6f75afb-1236-4caf-9f51-a8dc38604476","Type":"ContainerStarted","Data":"8610d88d43db21690e6ee02a62cb5288642efb85626d842b257df7401072c850"}
Jul 17 09:19:40.497998 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.497980    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:40.499542 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.499524    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j"
Jul 17 09:19:40.517732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.517691    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6dfdff4bcb-sj4zb" podStartSLOduration=1.783612013 podStartE2EDuration="5.517680141s" totalImagesPullingTime="3.734068128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:40.516413736 +0000 UTC m=+93.908406354" watchObservedRunningTime="2026-07-17 09:19:40.517680141 +0000 UTC m=+93.909672736"
Jul 17 09:19:40.554653 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:40.554614    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5fc44677d7-b7v2j" podStartSLOduration=1.777596261 podStartE2EDuration="5.554600235s" totalImagesPullingTime="3.777003974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:40.553753469 +0000 UTC m=+93.945746102" watchObservedRunningTime="2026-07-17 09:19:40.554600235 +0000 UTC m=+93.946592851"
Jul 17 09:19:41.233603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:41.233561    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:41.233603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:41.233621    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:41.233929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:41.233917    2575 scope.go:117] "RemoveContainer" containerID="89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1"
Jul 17 09:19:41.234071 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:41.234053    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-rpjth_openshift-console-operator(b3528b82-4841-475a-9e5b-5d94d683ac5c)\"" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" podUID="b3528b82-4841-475a-9e5b-5d94d683ac5c"
Jul 17 09:19:42.020224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:42.020187    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:42.020704 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:42.020247    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:42.020704 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:42.020327    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:42.020704 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:42.020370    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:42.020704 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:42.020380    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:50.020365417 +0000 UTC m=+103.412358012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:42.020704 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:42.020446    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:50.020428595 +0000 UTC m=+103.412421199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:45.850603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:45.850515    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:19:45.852769 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:45.852748    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc8a050f-5f64-4c67-b0a6-93d5877df02d-metrics-tls\") pod \"dns-default-4jtlg\" (UID: \"cc8a050f-5f64-4c67-b0a6-93d5877df02d\") " pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:19:45.911677 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:45.911651    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4mh5z\""
Jul 17 09:19:45.919230 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:45.919203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:19:46.043813 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:46.043787    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4jtlg"]
Jul 17 09:19:46.046749 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:19:46.046720    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8a050f_5f64_4c67_b0a6_93d5877df02d.slice/crio-9830d41cf1f0d0c7acd8413730167fdc6b7790a4c269338d358ba6daf154e20c WatchSource:0}: Error finding container 9830d41cf1f0d0c7acd8413730167fdc6b7790a4c269338d358ba6daf154e20c: Status 404 returned error can't find the container with id 9830d41cf1f0d0c7acd8413730167fdc6b7790a4c269338d358ba6daf154e20c
Jul 17 09:19:46.515376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:46.515341    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jtlg" event={"ID":"cc8a050f-5f64-4c67-b0a6-93d5877df02d","Type":"ContainerStarted","Data":"9830d41cf1f0d0c7acd8413730167fdc6b7790a4c269338d358ba6daf154e20c"}
Jul 17 09:19:46.757215 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:46.757182    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:46.760076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:46.760043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8e31e400-40be-4561-860f-7c83a2fa2da3-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-dsxvz\" (UID: \"8e31e400-40be-4561-860f-7c83a2fa2da3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:46.906871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:46.906794    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"
Jul 17 09:19:47.056757 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:47.056726    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz"]
Jul 17 09:19:47.519077 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:47.519049    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz" event={"ID":"8e31e400-40be-4561-860f-7c83a2fa2da3","Type":"ContainerStarted","Data":"1e5633d5d3aa2e04acf5f28c2bdd3ec0b545be235ce4fa19b3fd9b2f93a8f01f"}
Jul 17 09:19:48.525056 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:48.525022    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jtlg" event={"ID":"cc8a050f-5f64-4c67-b0a6-93d5877df02d","Type":"ContainerStarted","Data":"bfa1f4b1bc0e7aa08321b0d185c2d72002f3633a82bc8b145476f9496ca1491f"}
Jul 17 09:19:48.525520 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:48.525084    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4jtlg" event={"ID":"cc8a050f-5f64-4c67-b0a6-93d5877df02d","Type":"ContainerStarted","Data":"fb4bf85874a1b6b1993a536f7d22255b28ada8eb6794e0832f474759b60b006a"}
Jul 17 09:19:48.525520 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:48.525188    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:19:48.545997 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:48.545949    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4jtlg" podStartSLOduration=65.585542035 podStartE2EDuration="1m7.545930811s" totalImagesPullingTime="1.960388776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:48.545114351 +0000 UTC m=+101.937106970" watchObservedRunningTime="2026-07-17 09:19:48.545930811 +0000 UTC m=+101.937923429"
Jul 17 09:19:49.424343 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:49.424314    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zql5k"
Jul 17 09:19:50.085349 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:50.085314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:50.085722 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:50.085382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:50.085722 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:50.085479    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:50.085722 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:19:50.085532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls podName:c97aa36f-dada-4dba-ab4e-d7e3e6c18606 nodeName:}" failed. No retries permitted until 2026-07-17 09:20:06.085519207 +0000 UTC m=+119.477511803 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ls2cv" (UID: "c97aa36f-dada-4dba-ab4e-d7e3e6c18606") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:50.085830 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:50.085756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:19:50.533495 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:50.533459    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz" event={"ID":"8e31e400-40be-4561-860f-7c83a2fa2da3","Type":"ContainerStarted","Data":"0d1bd34b85ae661638251f2b2196de56c16493c9512121dcfa7098731046dcf0"}
Jul 17 09:19:51.538346 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:51.538293    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz" event={"ID":"8e31e400-40be-4561-860f-7c83a2fa2da3","Type":"ContainerStarted","Data":"6d14bf05a8344003109cd7f9747742daa2c43c095f9cfde3f8c9be2cf683fdac"}
Jul 17 09:19:51.569760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:51.569703    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-dsxvz" podStartSLOduration=18.306648863 podStartE2EDuration="21.569689429s" totalImagesPullingTime="3.263040566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:51.569252341 +0000 UTC m=+104.961244969" watchObservedRunningTime="2026-07-17 09:19:51.569689429 +0000 UTC m=+104.961682046"
Jul 17 09:19:53.147601 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:53.147570    2575 scope.go:117] "RemoveContainer" containerID="89e6c95dd22023283594d807ad8fa0dd447ec40ea6618b5eaf9c7bfe5425aea1"
Jul 17 09:19:53.546600 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:53.546575    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:19:53.546750 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:53.546654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" event={"ID":"b3528b82-4841-475a-9e5b-5d94d683ac5c","Type":"ContainerStarted","Data":"9e64e1295da1a0faecb0d0b3c9324aae5c7af56a34b4d311b097c848eb325044"}
Jul 17 09:19:53.546914 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:53.546882    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:53.579017 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:53.578975    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-rpjth" podStartSLOduration=20.645916973 podStartE2EDuration="23.578960155s" totalImagesPullingTime="2.933043182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:53.578407926 +0000 UTC m=+106.970400542" watchObservedRunningTime="2026-07-17 09:19:53.578960155 +0000 UTC m=+106.970952772"
Jul 17 09:19:54.164869 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:54.164841    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-rpjth"
Jul 17 09:19:58.531698 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:19:58.531665    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4jtlg"
Jul 17 09:20:05.289361 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.289328    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:20:05.292579 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.292562    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.296467 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296444    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:20:05.296597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296456    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:20:05.296597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296506    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:20:05.296597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296521    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:20:05.296597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296491    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wp7bs\""
Jul 17 09:20:05.296597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296589    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:20:05.296871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296795    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:20:05.296871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.296815    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:20:05.304064 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.304044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:20:05.399254 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4clc\" (UniqueName: \"kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.399388 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399265    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.399388 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.399470 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.399470 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.399470 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.399451    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.500356 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.500328    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4clc\" (UniqueName: \"kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.500490 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.500364    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.500745 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.500575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.500900 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.500881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.500986 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.500932    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.501574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.501542    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.501700 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.501658    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.501772 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.501705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.503726 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.502556    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.504121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.504100    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.507194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.507175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.511734 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.511710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4clc\" (UniqueName: \"kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc\") pod \"console-5dbb4989d-zg7ng\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") " pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.601547 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.601491    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:05.738428 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:05.738399    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:20:05.741668 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:05.741639    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0149b6e_5603_4638_8f23_a27c3663be9d.slice/crio-fac40c993bfdabb6458ccf5a50098edeff57c6a004329f77a3307276a6513937 WatchSource:0}: Error finding container fac40c993bfdabb6458ccf5a50098edeff57c6a004329f77a3307276a6513937: Status 404 returned error can't find the container with id fac40c993bfdabb6458ccf5a50098edeff57c6a004329f77a3307276a6513937
Jul 17 09:20:06.106934 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.106896    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:20:06.109249 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.109228    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c97aa36f-dada-4dba-ab4e-d7e3e6c18606-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ls2cv\" (UID: \"c97aa36f-dada-4dba-ab4e-d7e3e6c18606\") " pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:20:06.158624 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.158596    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ls2cv"
Jul 17 09:20:06.302191 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:06.302165    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc97aa36f_dada_4dba_ab4e_d7e3e6c18606.slice/crio-1848febb0c1ece25d6b2cbfa8f43d7026d571445123fdd8c129c5400965961f7 WatchSource:0}: Error finding container 1848febb0c1ece25d6b2cbfa8f43d7026d571445123fdd8c129c5400965961f7: Status 404 returned error can't find the container with id 1848febb0c1ece25d6b2cbfa8f43d7026d571445123fdd8c129c5400965961f7
Jul 17 09:20:06.323039 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.323012    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ls2cv"]
Jul 17 09:20:06.589023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.588989    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ls2cv" event={"ID":"c97aa36f-dada-4dba-ab4e-d7e3e6c18606","Type":"ContainerStarted","Data":"1848febb0c1ece25d6b2cbfa8f43d7026d571445123fdd8c129c5400965961f7"}
Jul 17 09:20:06.590810 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:06.590778    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dbb4989d-zg7ng" event={"ID":"a0149b6e-5603-4638-8f23-a27c3663be9d","Type":"ContainerStarted","Data":"fac40c993bfdabb6458ccf5a50098edeff57c6a004329f77a3307276a6513937"}
Jul 17 09:20:09.740847 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.740818    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"]
Jul 17 09:20:09.786910 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.786863    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"]
Jul 17 09:20:09.787044 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.786977    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:09.794545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.794229    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 17 09:20:09.794545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.794259    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 17 09:20:09.794545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.794270    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 09:20:09.794545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.794287    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-l8tl5\""
Jul 17 09:20:09.794791 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.794611    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 09:20:09.797379 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.797358    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 09:20:09.938497 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.938456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:09.938668 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.938507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mffk2\" (UniqueName: \"kubernetes.io/projected/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-kube-api-access-mffk2\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:09.938668 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.938569    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:09.938668 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:09.938622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.039068 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.038985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.039068 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.039038    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mffk2\" (UniqueName: \"kubernetes.io/projected/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-kube-api-access-mffk2\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.039286 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.039073    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.039286 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.039119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.039633 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:20:10.039604    2575 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 17 09:20:10.039779 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:20:10.039688    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls podName:a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0 nodeName:}" failed. No retries permitted until 2026-07-17 09:20:10.539663894 +0000 UTC m=+123.931656490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-k5srx" (UID: "a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0") : secret "prometheus-operator-tls" not found
Jul 17 09:20:10.039854 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.039830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.042172 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.042122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.075831 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.075803    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mffk2\" (UniqueName: \"kubernetes.io/projected/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-kube-api-access-mffk2\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.544401 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.544315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.546946 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.546914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-k5srx\" (UID: \"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:10.697202 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:10.697165    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"
Jul 17 09:20:12.123162 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:12.123036    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-k5srx"]
Jul 17 09:20:12.413668 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:12.413643    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dcc94f_1f40_472a_ad9a_4d7e658a4aa0.slice/crio-aec11edb4e69965627992a5ad7b32a0d4fd07e5e82e18d5903cbcd53f8cc4626 WatchSource:0}: Error finding container aec11edb4e69965627992a5ad7b32a0d4fd07e5e82e18d5903cbcd53f8cc4626: Status 404 returned error can't find the container with id aec11edb4e69965627992a5ad7b32a0d4fd07e5e82e18d5903cbcd53f8cc4626
Jul 17 09:20:12.615004 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:12.614036    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dbb4989d-zg7ng" event={"ID":"a0149b6e-5603-4638-8f23-a27c3663be9d","Type":"ContainerStarted","Data":"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"}
Jul 17 09:20:12.615579 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:12.615552    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx" event={"ID":"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0","Type":"ContainerStarted","Data":"aec11edb4e69965627992a5ad7b32a0d4fd07e5e82e18d5903cbcd53f8cc4626"}
Jul 17 09:20:12.649671 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:12.649631    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dbb4989d-zg7ng" podStartSLOduration=0.981249912 podStartE2EDuration="7.649616188s" totalImagesPullingTime="6.668366276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:12.648311329 +0000 UTC m=+126.040303936" watchObservedRunningTime="2026-07-17 09:20:12.649616188 +0000 UTC m=+126.041608806"
Jul 17 09:20:13.623079 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:13.622996    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ls2cv" event={"ID":"c97aa36f-dada-4dba-ab4e-d7e3e6c18606","Type":"ContainerStarted","Data":"e90c9ceb715783db8ff6a6f7e23a6db7a1842d4e4454ef1a8aad751250c5b2db"}
Jul 17 09:20:14.629200 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:14.629164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx" event={"ID":"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0","Type":"ContainerStarted","Data":"6c597ddbb188adfd7fd825317ca041618c2ce9dc7dc86e4d05a2911fdcd006d8"}
Jul 17 09:20:14.629710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:14.629227    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx" event={"ID":"a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0","Type":"ContainerStarted","Data":"5dcdfc20d105cae02cd44cb8b11ca93a39c43364a085ea8363820af6cecaf07b"}
Jul 17 09:20:14.688448 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:14.688397    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-k5srx" podStartSLOduration=3.990268006 podStartE2EDuration="5.68838061s" totalImagesPullingTime="1.698112604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:14.687913656 +0000 UTC m=+128.079906334" watchObservedRunningTime="2026-07-17 09:20:14.68838061 +0000 UTC m=+128.080373226"
Jul 17 09:20:15.601827 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:15.601799    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:15.602065 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:15.602036    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:15.606484 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:15.606467    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:15.637114 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:15.637091    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:20:16.638247 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:16.638174    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ls2cv" event={"ID":"c97aa36f-dada-4dba-ab4e-d7e3e6c18606","Type":"ContainerStarted","Data":"695ce06defb860ba0ce5e60248a754740aec3075460b7664325849f8d0cc3101"}
Jul 17 09:20:16.898863 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:16.898779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:20:16.901408 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:16.901382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc5779d7-9440-4f6f-b8e1-4ca12489f555-metrics-certs\") pod \"network-metrics-daemon-6pxs2\" (UID: \"cc5779d7-9440-4f6f-b8e1-4ca12489f555\") " pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:20:16.965917 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:16.965707    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2lf7\""
Jul 17 09:20:16.973174 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:16.973145    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6pxs2"
Jul 17 09:20:17.156498 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.156404    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6pxs2"]
Jul 17 09:20:17.162690 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:17.162662    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5779d7_9440_4f6f_b8e1_4ca12489f555.slice/crio-fdc07ebd8e99811fbdcf35cdbdb1beb738a6cdb3c07e5ce7da01278a139ac312 WatchSource:0}: Error finding container fdc07ebd8e99811fbdcf35cdbdb1beb738a6cdb3c07e5ce7da01278a139ac312: Status 404 returned error can't find the container with id fdc07ebd8e99811fbdcf35cdbdb1beb738a6cdb3c07e5ce7da01278a139ac312
Jul 17 09:20:17.610934 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.610897    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"]
Jul 17 09:20:17.615516 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.615495    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.621826 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.621801    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 17 09:20:17.622766 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.622560    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 17 09:20:17.622766 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.622596    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4vrzh\""
Jul 17 09:20:17.643277 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.643244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6pxs2" event={"ID":"cc5779d7-9440-4f6f-b8e1-4ca12489f555","Type":"ContainerStarted","Data":"fdc07ebd8e99811fbdcf35cdbdb1beb738a6cdb3c07e5ce7da01278a139ac312"}
Jul 17 09:20:17.648097 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.648072    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"]
Jul 17 09:20:17.800898 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.800866    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dh7wg"]
Jul 17 09:20:17.804643 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.804619    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807446    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0850f735-6a1a-46a1-8b5e-cdce30a3b214-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807669    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thqtj\" (UniqueName: \"kubernetes.io/projected/0850f735-6a1a-46a1-8b5e-cdce30a3b214-kube-api-access-thqtj\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807767    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rq7jd\""
Jul 17 09:20:17.807966 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.807958    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 09:20:17.808393 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.808088    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 09:20:17.808393 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.808213    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 09:20:17.909122 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909039    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-wtmp\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909122 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql759\" (UniqueName: \"kubernetes.io/projected/89bc3bc7-fc09-4617-9e2e-38db6e72e221-kube-api-access-ql759\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909122 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-textfile\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909137    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-accelerators-collector-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-tls\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909330    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-metrics-client-ca\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thqtj\" (UniqueName: \"kubernetes.io/projected/0850f735-6a1a-46a1-8b5e-cdce30a3b214-kube-api-access-thqtj\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.909422 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909410    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-root\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-sys\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:17.909739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909464    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.909739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0850f735-6a1a-46a1-8b5e-cdce30a3b214-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.909739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.909716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.910721 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.910682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0850f735-6a1a-46a1-8b5e-cdce30a3b214-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.912580 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.912557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.912690 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.912611    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0850f735-6a1a-46a1-8b5e-cdce30a3b214-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:17.933625 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:17.933595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thqtj\" (UniqueName: \"kubernetes.io/projected/0850f735-6a1a-46a1-8b5e-cdce30a3b214-kube-api-access-thqtj\") pod \"openshift-state-metrics-79f6958cdf-zk549\" (UID: \"0850f735-6a1a-46a1-8b5e-cdce30a3b214\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010332    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-wtmp\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql759\" (UniqueName: \"kubernetes.io/projected/89bc3bc7-fc09-4617-9e2e-38db6e72e221-kube-api-access-ql759\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010410    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-textfile\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010441    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-accelerators-collector-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-tls\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-metrics-client-ca\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010571    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-wtmp\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-root\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-sys\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010649    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-root\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.010753    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/89bc3bc7-fc09-4617-9e2e-38db6e72e221-sys\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011580 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.011191    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-accelerators-collector-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011580 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.011248    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-textfile\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.011701 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.011677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/89bc3bc7-fc09-4617-9e2e-38db6e72e221-metrics-client-ca\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.013437 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.013413    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.015549 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.015526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/89bc3bc7-fc09-4617-9e2e-38db6e72e221-node-exporter-tls\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.041806 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.041779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql759\" (UniqueName: \"kubernetes.io/projected/89bc3bc7-fc09-4617-9e2e-38db6e72e221-kube-api-access-ql759\") pod \"node-exporter-dh7wg\" (UID: \"89bc3bc7-fc09-4617-9e2e-38db6e72e221\") " pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.116849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.116808    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dh7wg"
Jul 17 09:20:18.227624 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.227427    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"
Jul 17 09:20:18.231745 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:18.231587    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bc3bc7_fc09_4617_9e2e_38db6e72e221.slice/crio-3965cf630e43d20ac089064e09e66fe4d4cd712f8315d7846f3117c0788410f3 WatchSource:0}: Error finding container 3965cf630e43d20ac089064e09e66fe4d4cd712f8315d7846f3117c0788410f3: Status 404 returned error can't find the container with id 3965cf630e43d20ac089064e09e66fe4d4cd712f8315d7846f3117c0788410f3
Jul 17 09:20:18.467964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.467933    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549"]
Jul 17 09:20:18.649349 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.649246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6pxs2" event={"ID":"cc5779d7-9440-4f6f-b8e1-4ca12489f555","Type":"ContainerStarted","Data":"a9c7668397ec1d61e79887753c9f0d15794d5d3f274c0c63923a2d717603331b"}
Jul 17 09:20:18.650655 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.650588    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dh7wg" event={"ID":"89bc3bc7-fc09-4617-9e2e-38db6e72e221","Type":"ContainerStarted","Data":"3965cf630e43d20ac089064e09e66fe4d4cd712f8315d7846f3117c0788410f3"}
Jul 17 09:20:18.846494 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.846462    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 09:20:18.857214 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.857186    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901818    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901816    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901816    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901924    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901951    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.901825    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 17 09:20:18.902197 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.902115    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 17 09:20:18.902728 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.902237    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 17 09:20:18.902728 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.902294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lk2zh\""
Jul 17 09:20:18.912466 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.912441    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 09:20:18.916573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:18.916549    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 17 09:20:19.019101 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019315 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019315 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019315 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019164    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019315 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019188    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019315 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019210    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019313    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019399    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019438    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019472    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrvj\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-kube-api-access-qhrvj\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019499    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019794 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.019794 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.019629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.119995 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.119958    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120007    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrvj\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-kube-api-access-qhrvj\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120130    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120475 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.120475 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120477    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.120493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.123275    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.123387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.123419    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.123680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.123768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.124048    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.125828 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.125416    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.130716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.126425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7f366ae-a012-4217-9b1e-ae395200fc9e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.130716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.127695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-volume\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.130716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.127714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7f366ae-a012-4217-9b1e-ae395200fc9e-config-out\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.130716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.129435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-web-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.131376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.131329    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.132990 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.132963    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.133554 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.133522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.134547 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.134513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.134911 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.134892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7f366ae-a012-4217-9b1e-ae395200fc9e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.159356 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.159280    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrvj\" (UniqueName: \"kubernetes.io/projected/f7f366ae-a012-4217-9b1e-ae395200fc9e-kube-api-access-qhrvj\") pod \"alertmanager-main-0\" (UID: \"f7f366ae-a012-4217-9b1e-ae395200fc9e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:19.167079 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:19.167055    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 17 09:20:20.089559 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:20.089531    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0850f735_6a1a_46a1_8b5e_cdce30a3b214.slice/crio-7b6a66f0e34bc92e8acc3ac272c9dcdeee206267da28c1cb1d3865fe89035f0a WatchSource:0}: Error finding container 7b6a66f0e34bc92e8acc3ac272c9dcdeee206267da28c1cb1d3865fe89035f0a: Status 404 returned error can't find the container with id 7b6a66f0e34bc92e8acc3ac272c9dcdeee206267da28c1cb1d3865fe89035f0a
Jul 17 09:20:20.291618 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.291592    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 17 09:20:20.297820 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:20.297790    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f366ae_a012_4217_9b1e_ae395200fc9e.slice/crio-16f8ef4e16cccb24951b2861a4c972239642ab01ab755c88ae62d772c5179045 WatchSource:0}: Error finding container 16f8ef4e16cccb24951b2861a4c972239642ab01ab755c88ae62d772c5179045: Status 404 returned error can't find the container with id 16f8ef4e16cccb24951b2861a4c972239642ab01ab755c88ae62d772c5179045
Jul 17 09:20:20.660284 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.660247    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ls2cv" event={"ID":"c97aa36f-dada-4dba-ab4e-d7e3e6c18606","Type":"ContainerStarted","Data":"06059dec332164aad4f49ecebc3f41718ad19442c1515ef9f2222f3fac78cb08"}
Jul 17 09:20:20.662215 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.662185    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549" event={"ID":"0850f735-6a1a-46a1-8b5e-cdce30a3b214","Type":"ContainerStarted","Data":"7b7445d07258e26b67141ee17a480d1c9f468019d543e1e9e00b0b0e5ff2ea5d"}
Jul 17 09:20:20.662365 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.662246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549" event={"ID":"0850f735-6a1a-46a1-8b5e-cdce30a3b214","Type":"ContainerStarted","Data":"7fb8ea19ee1d82ec3549dca136a6ebb6693b69d5ff0e285b53b5d7ccba16e96e"}
Jul 17 09:20:20.662365 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.662281    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549" event={"ID":"0850f735-6a1a-46a1-8b5e-cdce30a3b214","Type":"ContainerStarted","Data":"7b6a66f0e34bc92e8acc3ac272c9dcdeee206267da28c1cb1d3865fe89035f0a"}
Jul 17 09:20:20.663829 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.663798    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6pxs2" event={"ID":"cc5779d7-9440-4f6f-b8e1-4ca12489f555","Type":"ContainerStarted","Data":"6b494110b7a2422f21e5b227610111d6962f7a7deaca569af5452871ff74d65d"}
Jul 17 09:20:20.665082 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.665058    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"16f8ef4e16cccb24951b2861a4c972239642ab01ab755c88ae62d772c5179045"}
Jul 17 09:20:20.683153 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.683109    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ls2cv" podStartSLOduration=33.011929303 podStartE2EDuration="46.683093039s" totalImagesPullingTime="13.671163736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:19:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:20.682946124 +0000 UTC m=+134.074938742" watchObservedRunningTime="2026-07-17 09:20:20.683093039 +0000 UTC m=+134.075085706"
Jul 17 09:20:20.710498 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:20.710460    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6pxs2" podStartSLOduration=132.59942564 podStartE2EDuration="2m13.710444719s" totalImagesPullingTime="1.111019079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:20.709001183 +0000 UTC m=+134.100993799" watchObservedRunningTime="2026-07-17 09:20:20.710444719 +0000 UTC m=+134.102437374"
Jul 17 09:20:21.670135 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:21.670105    2575 generic.go:356] "Generic (PLEG): container finished" podID="89bc3bc7-fc09-4617-9e2e-38db6e72e221" containerID="eb49405cab75f7ffe0443838ab128fa27efc23cc6d3c4ffd39370f3fd934f139" exitCode=0
Jul 17 09:20:21.670516 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:21.670193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dh7wg" event={"ID":"89bc3bc7-fc09-4617-9e2e-38db6e72e221","Type":"ContainerDied","Data":"eb49405cab75f7ffe0443838ab128fa27efc23cc6d3c4ffd39370f3fd934f139"}
Jul 17 09:20:22.339433 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.339401    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78b4458c64-qk8k2"]
Jul 17 09:20:22.342412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.342395    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.345727 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.345671    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 17 09:20:22.346687 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.346669    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 17 09:20:22.346799 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.346688    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 09:20:22.346799 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.346748    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 17 09:20:22.346900 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.346813    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xjd7k\""
Jul 17 09:20:22.347014 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.347001    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4k1n5bh8au59a\""
Jul 17 09:20:22.358597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.358572    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b4458c64-qk8k2"]
Jul 17 09:20:22.453409 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453535 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-tls\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453535 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-metrics-server-audit-profiles\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453535 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453505    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-client-certs\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453645 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453553    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w596c\" (UniqueName: \"kubernetes.io/projected/853093a6-be68-4b1b-811d-7e061320b691-kube-api-access-w596c\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453645 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/853093a6-be68-4b1b-811d-7e061320b691-audit-log\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.453706 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.453650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-client-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554735 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-client-certs\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w596c\" (UniqueName: \"kubernetes.io/projected/853093a6-be68-4b1b-811d-7e061320b691-kube-api-access-w596c\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/853093a6-be68-4b1b-811d-7e061320b691-audit-log\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554789    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-client-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554826    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.554867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-tls\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.555115 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.554871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-metrics-server-audit-profiles\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.555362 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.555330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/853093a6-be68-4b1b-811d-7e061320b691-audit-log\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.555689 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.555669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.555914 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.555897    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/853093a6-be68-4b1b-811d-7e061320b691-metrics-server-audit-profiles\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.557260 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.557236    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-client-certs\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.557697 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.557677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-client-ca-bundle\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.557818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.557797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/853093a6-be68-4b1b-811d-7e061320b691-secret-metrics-server-tls\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.571833 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.571815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w596c\" (UniqueName: \"kubernetes.io/projected/853093a6-be68-4b1b-811d-7e061320b691-kube-api-access-w596c\") pod \"metrics-server-78b4458c64-qk8k2\" (UID: \"853093a6-be68-4b1b-811d-7e061320b691\") " pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.652053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.651995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:22.676532 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.676500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dh7wg" event={"ID":"89bc3bc7-fc09-4617-9e2e-38db6e72e221","Type":"ContainerStarted","Data":"2b8812753e9cbdf76bb9130940bdfc2755b017c2ace7d382b48230eef967104f"}
Jul 17 09:20:22.676876 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.676563    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dh7wg" event={"ID":"89bc3bc7-fc09-4617-9e2e-38db6e72e221","Type":"ContainerStarted","Data":"54b960e87c8d90d032c4109cafb8257aac8f2a1cf714ce0b75390c6f0d9fc49c"}
Jul 17 09:20:22.678593 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.678568    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549" event={"ID":"0850f735-6a1a-46a1-8b5e-cdce30a3b214","Type":"ContainerStarted","Data":"11756bfe185a9716e07031aab2a8f74b331d3b9b498ae01ab551bf351517a4d6"}
Jul 17 09:20:22.680361 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.680340    2575 generic.go:356] "Generic (PLEG): container finished" podID="f7f366ae-a012-4217-9b1e-ae395200fc9e" containerID="441545be2a6a64043782bca29f651a3b11a24ab000f95e7781bf6109c874bdcc" exitCode=0
Jul 17 09:20:22.680459 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.680365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerDied","Data":"441545be2a6a64043782bca29f651a3b11a24ab000f95e7781bf6109c874bdcc"}
Jul 17 09:20:22.713269 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.713225    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dh7wg" podStartSLOduration=3.190432793 podStartE2EDuration="5.713209065s" totalImagesPullingTime="2.457224479s" totalInitContainerRuntime="65.551793ms" isStatefulPod=true podCreationTimestamp="2026-07-17 09:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:22.712570167 +0000 UTC m=+136.104562787" watchObservedRunningTime="2026-07-17 09:20:22.713209065 +0000 UTC m=+136.105201660"
Jul 17 09:20:22.746954 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.746905    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zk549" podStartSLOduration=4.348558105 podStartE2EDuration="5.746886071s" totalImagesPullingTime="1.398327966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:22.74601736 +0000 UTC m=+136.138009990" watchObservedRunningTime="2026-07-17 09:20:22.746886071 +0000 UTC m=+136.138878689"
Jul 17 09:20:22.803634 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:22.803549    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b4458c64-qk8k2"]
Jul 17 09:20:22.805720 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:20:22.805695    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853093a6_be68_4b1b_811d_7e061320b691.slice/crio-777a897d5992a161c8762aa31464c6f452366a2fce345172a4fb21168f0cd203 WatchSource:0}: Error finding container 777a897d5992a161c8762aa31464c6f452366a2fce345172a4fb21168f0cd203: Status 404 returned error can't find the container with id 777a897d5992a161c8762aa31464c6f452366a2fce345172a4fb21168f0cd203
Jul 17 09:20:23.685664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:23.685619    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2" event={"ID":"853093a6-be68-4b1b-811d-7e061320b691","Type":"ContainerStarted","Data":"777a897d5992a161c8762aa31464c6f452366a2fce345172a4fb21168f0cd203"}
Jul 17 09:20:25.696922 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.696881    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"7423e2ecf02df80989c9f6d587824aba201711fddc7801248caafd9527523e3b"}
Jul 17 09:20:25.697376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.696966    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"20427244b08f34b942f5c760355dcb4df70de57f5ebe55711cdb81e753e37adf"}
Jul 17 09:20:25.697376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.697009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"46d933605c8d44b13022273b852c3193be8fe8ac9f3e5316629f87556096a99e"}
Jul 17 09:20:25.697376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.697086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"4517c5738fad0df2c94e1ae81845139b2c1ff95cf8305a415bedc00b7395042e"}
Jul 17 09:20:25.697376 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.697124    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"610918dbca0eec388838dabd888d6e8a1b2518e87beec598cc3c0920e9302c64"}
Jul 17 09:20:25.698413 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.698381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2" event={"ID":"853093a6-be68-4b1b-811d-7e061320b691","Type":"ContainerStarted","Data":"62a7797cd89572bfe7961d4f7f59715bef76ce675b63404f8459c7e7649bad96"}
Jul 17 09:20:25.725201 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:25.725158    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2" podStartSLOduration=1.7264935289999999 podStartE2EDuration="3.725139453s" totalImagesPullingTime="1.998645924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:25.72294507 +0000 UTC m=+139.114937688" watchObservedRunningTime="2026-07-17 09:20:25.725139453 +0000 UTC m=+139.117132125"
Jul 17 09:20:26.706070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:26.706033    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f7f366ae-a012-4217-9b1e-ae395200fc9e","Type":"ContainerStarted","Data":"a6316f3b53339508bc5e85e1a812ee8d6129ef33fdaa9ec0db802da1f14c3fff"}
Jul 17 09:20:26.738824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:26.738769    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.299655536 podStartE2EDuration="8.738749302s" totalImagesPullingTime="4.414708232s" totalInitContainerRuntime="24.385534ms" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:26.736760879 +0000 UTC m=+140.128753499" watchObservedRunningTime="2026-07-17 09:20:26.738749302 +0000 UTC m=+140.130741920"
Jul 17 09:20:34.857537 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:34.857495    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:20:42.652817 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:42.652784    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:42.653195 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:42.652842    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:20:55.819957 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:55.819923    2575 generic.go:356] "Generic (PLEG): container finished" podID="a9d6b919-6402-4646-ab7f-0dfe1e440e49" containerID="c25779bc7aa4ec4655a14eb036b637693c6b70c592d865ebf5e6f7c393d52611" exitCode=0
Jul 17 09:20:55.820410 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:55.819997    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr" event={"ID":"a9d6b919-6402-4646-ab7f-0dfe1e440e49","Type":"ContainerDied","Data":"c25779bc7aa4ec4655a14eb036b637693c6b70c592d865ebf5e6f7c393d52611"}
Jul 17 09:20:55.820410 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:55.820279    2575 scope.go:117] "RemoveContainer" containerID="c25779bc7aa4ec4655a14eb036b637693c6b70c592d865ebf5e6f7c393d52611"
Jul 17 09:20:56.826119 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:56.826083    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-759tr" event={"ID":"a9d6b919-6402-4646-ab7f-0dfe1e440e49","Type":"ContainerStarted","Data":"c28fda258aba02462a2408d7f5c704885e872b30148fc2156ec147aea0ef1398"}
Jul 17 09:20:59.877628 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:20:59.877542    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dbb4989d-zg7ng" podUID="a0149b6e-5603-4638-8f23-a27c3663be9d" containerName="console" containerID="cri-o://e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47" gracePeriod=15
Jul 17 09:21:00.134261 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.134204    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:21:00.256876 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.256841    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.256898    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.256945    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.256972    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4clc\" (UniqueName: \"kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.256995    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.257052    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config\") pod \"a0149b6e-5603-4638-8f23-a27c3663be9d\" (UID: \"a0149b6e-5603-4638-8f23-a27c3663be9d\") "
Jul 17 09:21:00.257394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.257364    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:21:00.257902 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.257807    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca" (OuterVolumeSpecName: "service-ca") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:21:00.257902 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.257860    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config" (OuterVolumeSpecName: "console-config") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:21:00.263647 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.263617    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:21:00.263793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.263776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:21:00.263875 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.263812    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc" (OuterVolumeSpecName: "kube-api-access-c4clc") pod "a0149b6e-5603-4638-8f23-a27c3663be9d" (UID: "a0149b6e-5603-4638-8f23-a27c3663be9d"). InnerVolumeSpecName "kube-api-access-c4clc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:21:00.358664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358630    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.358664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358658    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-oauth-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.358664 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358668    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.358867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358678    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4clc\" (UniqueName: \"kubernetes.io/projected/a0149b6e-5603-4638-8f23-a27c3663be9d-kube-api-access-c4clc\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.358867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358687    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0149b6e-5603-4638-8f23-a27c3663be9d-console-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.358867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.358695    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0149b6e-5603-4638-8f23-a27c3663be9d-console-oauth-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:21:00.839895 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.839861    2575 generic.go:356] "Generic (PLEG): container finished" podID="a0149b6e-5603-4638-8f23-a27c3663be9d" containerID="e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47" exitCode=0
Jul 17 09:21:00.840055 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.839917    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dbb4989d-zg7ng"
Jul 17 09:21:00.840055 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.839944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dbb4989d-zg7ng" event={"ID":"a0149b6e-5603-4638-8f23-a27c3663be9d","Type":"ContainerDied","Data":"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"}
Jul 17 09:21:00.840055 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.840014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dbb4989d-zg7ng" event={"ID":"a0149b6e-5603-4638-8f23-a27c3663be9d","Type":"ContainerDied","Data":"fac40c993bfdabb6458ccf5a50098edeff57c6a004329f77a3307276a6513937"}
Jul 17 09:21:00.840055 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.840035    2575 scope.go:117] "RemoveContainer" containerID="e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"
Jul 17 09:21:00.848502 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.848486    2575 scope.go:117] "RemoveContainer" containerID="e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"
Jul 17 09:21:00.848747 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:21:00.848730    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47\": container with ID starting with e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47 not found: ID does not exist" containerID="e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"
Jul 17 09:21:00.848791 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.848756    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47"} err="failed to get container status \"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47\": rpc error: code = NotFound desc = could not find container \"e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47\": container with ID starting with e36206dd48b13c4bef5a659fc79edc755334b8692063b89b8c03e427724ebe47 not found: ID does not exist"
Jul 17 09:21:00.864153 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.864133    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:21:00.870792 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:00.870772    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dbb4989d-zg7ng"]
Jul 17 09:21:01.152220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:01.152146    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0149b6e-5603-4638-8f23-a27c3663be9d" path="/var/lib/kubelet/pods/a0149b6e-5603-4638-8f23-a27c3663be9d/volumes"
Jul 17 09:21:02.658398 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:02.658368    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:21:02.662103 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:02.662082    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78b4458c64-qk8k2"
Jul 17 09:21:04.856818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:04.856736    2575 generic.go:356] "Generic (PLEG): container finished" podID="2ff6e71b-3867-44d1-a167-4625c3d3cd25" containerID="1553a6fe269b69726e2d4a3449128e52a0af15853d7aa5bdd6ca646549630687" exitCode=0
Jul 17 09:21:04.857252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:04.856812    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6" event={"ID":"2ff6e71b-3867-44d1-a167-4625c3d3cd25","Type":"ContainerDied","Data":"1553a6fe269b69726e2d4a3449128e52a0af15853d7aa5bdd6ca646549630687"}
Jul 17 09:21:04.857252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:04.857207    2575 scope.go:117] "RemoveContainer" containerID="1553a6fe269b69726e2d4a3449128e52a0af15853d7aa5bdd6ca646549630687"
Jul 17 09:21:05.863120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:05.863071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jxhk6" event={"ID":"2ff6e71b-3867-44d1-a167-4625c3d3cd25","Type":"ContainerStarted","Data":"187c237633c740883e985caf95afd6917a537903c899a3129ce7694bbd026b66"}
Jul 17 09:21:42.497675 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.497640    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"]
Jul 17 09:21:42.498155 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.497991    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0149b6e-5603-4638-8f23-a27c3663be9d" containerName="console"
Jul 17 09:21:42.498155 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.498003    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0149b6e-5603-4638-8f23-a27c3663be9d" containerName="console"
Jul 17 09:21:42.498155 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.498054    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0149b6e-5603-4638-8f23-a27c3663be9d" containerName="console"
Jul 17 09:21:42.501365 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.501344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.504113 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.504081    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 17 09:21:42.504224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.504164    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 17 09:21:42.504224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.504211    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 17 09:21:42.504735 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.504717    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zr598\""
Jul 17 09:21:42.505224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.505209    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 17 09:21:42.505327 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.505215    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 17 09:21:42.510572 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.510546    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 17 09:21:42.512558 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.512537    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"]
Jul 17 09:21:42.574693 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574693 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574690    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-metrics-client-ca\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-serving-certs-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574871 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-federate-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574967 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prjnc\" (UniqueName: \"kubernetes.io/projected/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-kube-api-access-prjnc\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574967 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.574967 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.574920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-metrics-client-ca\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-serving-certs-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.675862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-federate-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.676079 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prjnc\" (UniqueName: \"kubernetes.io/projected/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-kube-api-access-prjnc\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.676079 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.675952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.676603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.676577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-metrics-client-ca\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.676734 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.676619    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.676925 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.676902    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-serving-certs-ca-bundle\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.678540 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.678522    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-telemeter-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.678627 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.678548    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-federate-client-tls\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.678866 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.678846    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.678901 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.678850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-secret-telemeter-client\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.684562 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.684542    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prjnc\" (UniqueName: \"kubernetes.io/projected/a84b09f9-6fab-4186-b7fb-59c8e17d6bd0-kube-api-access-prjnc\") pod \"telemeter-client-7656dbdc6c-dg6l4\" (UID: \"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0\") " pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.812242 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.812180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"
Jul 17 09:21:42.949687 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:42.949546    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4"]
Jul 17 09:21:42.951999 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:21:42.951975    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84b09f9_6fab_4186_b7fb_59c8e17d6bd0.slice/crio-d1b2c95050bfcf50b542eff689735ced167083660c6572cd31324f847d83c0e7 WatchSource:0}: Error finding container d1b2c95050bfcf50b542eff689735ced167083660c6572cd31324f847d83c0e7: Status 404 returned error can't find the container with id d1b2c95050bfcf50b542eff689735ced167083660c6572cd31324f847d83c0e7
Jul 17 09:21:43.006754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:43.006726    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4" event={"ID":"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0","Type":"ContainerStarted","Data":"d1b2c95050bfcf50b542eff689735ced167083660c6572cd31324f847d83c0e7"}
Jul 17 09:21:45.016929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.016895    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4" event={"ID":"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0","Type":"ContainerStarted","Data":"52db09863e6cc3f87cac8135ce8997aecf7d8c8a5a55763360b7c51e72e8d168"}
Jul 17 09:21:45.017272 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.016957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4" event={"ID":"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0","Type":"ContainerStarted","Data":"53a2bb5103851e025076a2b739367cc654755c62d30e52560384930ae79e3ae8"}
Jul 17 09:21:45.017272 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.016984    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4" event={"ID":"a84b09f9-6fab-4186-b7fb-59c8e17d6bd0","Type":"ContainerStarted","Data":"175eba764d23696a1fa97f3c29dad48c449c2548d8c55321f9758ce12af44881"}
Jul 17 09:21:45.042259 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.042176    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7656dbdc6c-dg6l4" podStartSLOduration=1.215437296 podStartE2EDuration="3.042162564s" totalImagesPullingTime="1.826725268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:21:45.042039363 +0000 UTC m=+218.434032019" watchObservedRunningTime="2026-07-17 09:21:45.042162564 +0000 UTC m=+218.434155181"
Jul 17 09:21:45.803640 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.803552    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:21:45.807191 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.807165    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.810007 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.809983    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:21:45.811227 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811202    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:21:45.811374 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811245    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:21:45.811489 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811469    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:21:45.811489 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811483    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wp7bs\""
Jul 17 09:21:45.811621 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811501    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:21:45.811621 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811515    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:21:45.811621 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.811616    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:21:45.816136 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.816117    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 09:21:45.820065 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.820044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:21:45.903690 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903648    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.903894 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.903894 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903761    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.903894 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903793    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.903894 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.904102 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:45.904102 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:45.903928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvngv\" (UniqueName: \"kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004380 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004438    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004457    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.004827 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.004587    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvngv\" (UniqueName: \"kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.005283 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.005256    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.005455 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.005392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.005455 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.005392    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.005455 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.005445    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.007418 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.007398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.007540 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.007518    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.014746 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.014725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvngv\" (UniqueName: \"kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv\") pod \"console-84bb47c487-k67hp\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") " pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.118755 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.118693    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:21:46.256223 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:46.256188    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:21:46.259252 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:21:46.259221    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24fb20f_caa3_452b_8d28_26474c8a2d95.slice/crio-c75a5cb7137c9fca797265a52583f299c01cb2f83d50e8bad167d60a12a69d31 WatchSource:0}: Error finding container c75a5cb7137c9fca797265a52583f299c01cb2f83d50e8bad167d60a12a69d31: Status 404 returned error can't find the container with id c75a5cb7137c9fca797265a52583f299c01cb2f83d50e8bad167d60a12a69d31
Jul 17 09:21:47.026969 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:47.026936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb47c487-k67hp" event={"ID":"f24fb20f-caa3-452b-8d28-26474c8a2d95","Type":"ContainerStarted","Data":"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"}
Jul 17 09:21:47.027146 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:47.026982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb47c487-k67hp" event={"ID":"f24fb20f-caa3-452b-8d28-26474c8a2d95","Type":"ContainerStarted","Data":"c75a5cb7137c9fca797265a52583f299c01cb2f83d50e8bad167d60a12a69d31"}
Jul 17 09:21:47.059204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:47.059154    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84bb47c487-k67hp" podStartSLOduration=2.059141438 podStartE2EDuration="2.059141438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:21:47.055404727 +0000 UTC m=+220.447397354" watchObservedRunningTime="2026-07-17 09:21:47.059141438 +0000 UTC m=+220.451134055"
Jul 17 09:21:54.375798 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.375756    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:21:54.408777 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.408751    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:21:54.412145 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.412128    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.422600 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.422580    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:21:54.578086 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578086 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578084    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqp8\" (UniqueName: \"kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.578280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.578260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.678883 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678807    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.678883 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.678883 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678921    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqp8\" (UniqueName: \"kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.678977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679690 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.679664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679795 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.679664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679856 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.679784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.679856 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.679822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.681313 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.681281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.681545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.681528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.688194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.688157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqp8\" (UniqueName: \"kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8\") pod \"console-85d4cd9497-lr2sn\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") " pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.721170 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.721151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:21:54.872760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:54.872722    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:21:54.881861 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:21:54.881833    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc979de2_4f08_45ec_a00f_dd6ceab4bdf3.slice/crio-510d50aff3e51e8381df852cf7eb5d617a52951483ff21fa5b8f70a3b48c47e1 WatchSource:0}: Error finding container 510d50aff3e51e8381df852cf7eb5d617a52951483ff21fa5b8f70a3b48c47e1: Status 404 returned error can't find the container with id 510d50aff3e51e8381df852cf7eb5d617a52951483ff21fa5b8f70a3b48c47e1
Jul 17 09:21:55.058586 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:55.058545    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4cd9497-lr2sn" event={"ID":"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3","Type":"ContainerStarted","Data":"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"}
Jul 17 09:21:55.058738 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:55.058596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4cd9497-lr2sn" event={"ID":"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3","Type":"ContainerStarted","Data":"510d50aff3e51e8381df852cf7eb5d617a52951483ff21fa5b8f70a3b48c47e1"}
Jul 17 09:21:55.093238 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:55.093194    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d4cd9497-lr2sn" podStartSLOduration=1.093179275 podStartE2EDuration="1.093179275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:21:55.091401798 +0000 UTC m=+228.483394414" watchObservedRunningTime="2026-07-17 09:21:55.093179275 +0000 UTC m=+228.485171891"
Jul 17 09:21:56.119267 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:21:56.119233    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:22:04.722263 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:04.722231    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:22:04.722703 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:04.722290    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:22:04.727189 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:04.727166    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:22:05.100707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:05.100604    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:22:19.395687 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.395636    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84bb47c487-k67hp" podUID="f24fb20f-caa3-452b-8d28-26474c8a2d95" containerName="console" containerID="cri-o://9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528" gracePeriod=15
Jul 17 09:22:19.634629 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.634603    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:22:19.673029 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.672968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673029 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673006    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvngv\" (UniqueName: \"kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673029 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673027    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673151    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673221    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673389 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673257    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673389 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673332    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle\") pod \"f24fb20f-caa3-452b-8d28-26474c8a2d95\" (UID: \"f24fb20f-caa3-452b-8d28-26474c8a2d95\") "
Jul 17 09:22:19.673822 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673567    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca" (OuterVolumeSpecName: "service-ca") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:22:19.673822 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673575    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config" (OuterVolumeSpecName: "console-config") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:22:19.673822 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673746    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:22:19.674048 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.673987    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:22:19.675264 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.675239    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:22:19.675397 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.675269    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv" (OuterVolumeSpecName: "kube-api-access-rvngv") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "kube-api-access-rvngv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:22:19.675645 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.675624    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f24fb20f-caa3-452b-8d28-26474c8a2d95" (UID: "f24fb20f-caa3-452b-8d28-26474c8a2d95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:22:19.774661 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774633    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774661 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774658    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774666    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774675    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-trusted-ca-bundle\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774683    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f24fb20f-caa3-452b-8d28-26474c8a2d95-console-oauth-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774691    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvngv\" (UniqueName: \"kubernetes.io/projected/f24fb20f-caa3-452b-8d28-26474c8a2d95-kube-api-access-rvngv\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:19.774797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:19.774699    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f24fb20f-caa3-452b-8d28-26474c8a2d95-oauth-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:22:20.154988 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.154950    2575 generic.go:356] "Generic (PLEG): container finished" podID="f24fb20f-caa3-452b-8d28-26474c8a2d95" containerID="9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528" exitCode=0
Jul 17 09:22:20.155160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.155002    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84bb47c487-k67hp"
Jul 17 09:22:20.155160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.155030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb47c487-k67hp" event={"ID":"f24fb20f-caa3-452b-8d28-26474c8a2d95","Type":"ContainerDied","Data":"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"}
Jul 17 09:22:20.155160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.155077    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84bb47c487-k67hp" event={"ID":"f24fb20f-caa3-452b-8d28-26474c8a2d95","Type":"ContainerDied","Data":"c75a5cb7137c9fca797265a52583f299c01cb2f83d50e8bad167d60a12a69d31"}
Jul 17 09:22:20.155160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.155096    2575 scope.go:117] "RemoveContainer" containerID="9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"
Jul 17 09:22:20.163597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.163581    2575 scope.go:117] "RemoveContainer" containerID="9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"
Jul 17 09:22:20.163836 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:22:20.163820    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528\": container with ID starting with 9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528 not found: ID does not exist" containerID="9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"
Jul 17 09:22:20.163877 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.163843    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528"} err="failed to get container status \"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528\": rpc error: code = NotFound desc = could not find container \"9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528\": container with ID starting with 9eb73748fdc70ac68a5d1cf755d6e27bff8ebc8f3106beef7f4f278d6e6a7528 not found: ID does not exist"
Jul 17 09:22:20.184928 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.184907    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:22:20.193401 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:20.193381    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84bb47c487-k67hp"]
Jul 17 09:22:21.152214 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:22:21.152181    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f24fb20f-caa3-452b-8d28-26474c8a2d95" path="/var/lib/kubelet/pods/f24fb20f-caa3-452b-8d28-26474c8a2d95/volumes"
Jul 17 09:23:07.037060 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:07.037035    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:23:07.042621 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:07.042602    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:23:07.048410 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:07.048392    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 09:23:07.053159 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:07.053144    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:23:07.059312 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:07.059282    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:23:53.537535 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.537498    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"]
Jul 17 09:23:53.538080 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.537918    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f24fb20f-caa3-452b-8d28-26474c8a2d95" containerName="console"
Jul 17 09:23:53.538080 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.537937    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f24fb20f-caa3-452b-8d28-26474c8a2d95" containerName="console"
Jul 17 09:23:53.538080 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.538041    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f24fb20f-caa3-452b-8d28-26474c8a2d95" containerName="console"
Jul 17 09:23:53.540947 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.540928    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.543813 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.543790    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 09:23:53.543940 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.543840    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 09:23:53.544740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.544725    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lhw7m\""
Jul 17 09:23:53.554364 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.554338    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"]
Jul 17 09:23:53.620802 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.620773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2phd\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-kube-api-access-j2phd\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.620929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.620811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.721569 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.721537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2phd\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-kube-api-access-j2phd\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.721678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.721584    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.734704 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.734677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.734907 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.734883    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2phd\" (UniqueName: \"kubernetes.io/projected/42c9f001-377c-4000-b5d5-da9073578353-kube-api-access-j2phd\") pod \"cert-manager-cainjector-559cdd8c87-9chmv\" (UID: \"42c9f001-377c-4000-b5d5-da9073578353\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.850007 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.849942    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"
Jul 17 09:23:53.990807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.990785    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9chmv"]
Jul 17 09:23:53.993431 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:23:53.993403    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42c9f001_377c_4000_b5d5_da9073578353.slice/crio-b8f76681a3cbd4d0d70381867e853be028a81b6c29ad41e38db802e20ed532d5 WatchSource:0}: Error finding container b8f76681a3cbd4d0d70381867e853be028a81b6c29ad41e38db802e20ed532d5: Status 404 returned error can't find the container with id b8f76681a3cbd4d0d70381867e853be028a81b6c29ad41e38db802e20ed532d5
Jul 17 09:23:53.995697 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:53.995681    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:23:54.509282 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:54.509240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv" event={"ID":"42c9f001-377c-4000-b5d5-da9073578353","Type":"ContainerStarted","Data":"b8f76681a3cbd4d0d70381867e853be028a81b6c29ad41e38db802e20ed532d5"}
Jul 17 09:23:57.524824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:57.524787    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv" event={"ID":"42c9f001-377c-4000-b5d5-da9073578353","Type":"ContainerStarted","Data":"03bab22c7dd424a2d376b52bb6adc00f046a09b9a051bbc0b3253fa8711290d9"}
Jul 17 09:23:57.544709 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:23:57.544661    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9chmv" podStartSLOduration=1.7204773580000001 podStartE2EDuration="4.544645882s" totalImagesPullingTime="2.824168524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:23:57.54368373 +0000 UTC m=+350.935676349" watchObservedRunningTime="2026-07-17 09:23:57.544645882 +0000 UTC m=+350.936638499"
Jul 17 09:24:07.284419 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.284346    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"]
Jul 17 09:24:07.287479 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.287464    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.290390 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.290370    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:24:07.291514 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.291495    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 09:24:07.291616 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.291500    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tv27j\""
Jul 17 09:24:07.299795 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.299776    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"]
Jul 17 09:24:07.332516 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.332492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.332612 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.332593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd9lw\" (UniqueName: \"kubernetes.io/projected/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-kube-api-access-hd9lw\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.433009 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.432987    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd9lw\" (UniqueName: \"kubernetes.io/projected/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-kube-api-access-hd9lw\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.433098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.433018    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.433443 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.433426    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.444171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.444144    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd9lw\" (UniqueName: \"kubernetes.io/projected/30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d-kube-api-access-hd9lw\") pod \"openshift-lws-operator-fd8ccff4c-bk2r2\" (UID: \"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.597036 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.596972    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"
Jul 17 09:24:07.728754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:07.728733    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2"]
Jul 17 09:24:07.731244 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:24:07.731217    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30bf22d5_0cd5_4d3a_81ab_7f0df9a55c5d.slice/crio-af52de2adae2b2527336879b0bed4a0261b81f372f7860efb237aa9697ce82f0 WatchSource:0}: Error finding container af52de2adae2b2527336879b0bed4a0261b81f372f7860efb237aa9697ce82f0: Status 404 returned error can't find the container with id af52de2adae2b2527336879b0bed4a0261b81f372f7860efb237aa9697ce82f0
Jul 17 09:24:08.572352 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:08.572181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2" event={"ID":"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d","Type":"ContainerStarted","Data":"af52de2adae2b2527336879b0bed4a0261b81f372f7860efb237aa9697ce82f0"}
Jul 17 09:24:10.581881 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:10.581846    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2" event={"ID":"30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d","Type":"ContainerStarted","Data":"5310da8b3049c2ab762f416665954fe1c8b1dd3e2e925c3cd57d193a911010cc"}
Jul 17 09:24:10.601375 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:10.601329    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-bk2r2" podStartSLOduration=1.236837301 podStartE2EDuration="3.601312113s" totalImagesPullingTime="2.364474812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:10.599567164 +0000 UTC m=+363.991559781" watchObservedRunningTime="2026-07-17 09:24:10.601312113 +0000 UTC m=+363.993304725"
Jul 17 09:24:29.859718 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.859684    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"]
Jul 17 09:24:29.864575 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.864552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:29.872675 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.872653    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 09:24:29.874288 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.874272    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 09:24:29.875011 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.874989    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 09:24:29.875325 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.875285    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pljhm\""
Jul 17 09:24:29.875530 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.875518    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 09:24:29.880850 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.880831    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"]
Jul 17 09:24:29.904589 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.904562    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:29.904682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.904613    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:29.904682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.904661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jstb\" (UniqueName: \"kubernetes.io/projected/718156a7-f6a4-4e41-8b63-56c317205082-kube-api-access-6jstb\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:29.904755 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.904699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:29.904788 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:29.904762    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.005951 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.005919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.006120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.005960    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jstb\" (UniqueName: \"kubernetes.io/projected/718156a7-f6a4-4e41-8b63-56c317205082-kube-api-access-6jstb\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.006120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.005989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.006120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.006019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.006120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.006108    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.006400 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.006378    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.008112 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.008084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/718156a7-f6a4-4e41-8b63-56c317205082-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.008348 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.008331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.008527 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.008506    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/718156a7-f6a4-4e41-8b63-56c317205082-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.027572 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.027547    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jstb\" (UniqueName: \"kubernetes.io/projected/718156a7-f6a4-4e41-8b63-56c317205082-kube-api-access-6jstb\") pod \"opendatahub-operator-controller-manager-76fdbb8988-rhjtm\" (UID: \"718156a7-f6a4-4e41-8b63-56c317205082\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.175348 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.175237    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:30.345015 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.344947    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"]
Jul 17 09:24:30.348044 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:24:30.348019    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod718156a7_f6a4_4e41_8b63_56c317205082.slice/crio-b0bcf6ced2916ae9901580b3213a4b358f65f50dceb17ec3ffd72ac05493ceb7 WatchSource:0}: Error finding container b0bcf6ced2916ae9901580b3213a4b358f65f50dceb17ec3ffd72ac05493ceb7: Status 404 returned error can't find the container with id b0bcf6ced2916ae9901580b3213a4b358f65f50dceb17ec3ffd72ac05493ceb7
Jul 17 09:24:30.657261 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:30.657226    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm" event={"ID":"718156a7-f6a4-4e41-8b63-56c317205082","Type":"ContainerStarted","Data":"b0bcf6ced2916ae9901580b3213a4b358f65f50dceb17ec3ffd72ac05493ceb7"}
Jul 17 09:24:33.672824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:33.672737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm" event={"ID":"718156a7-f6a4-4e41-8b63-56c317205082","Type":"ContainerStarted","Data":"48b2e98b8402bc7f51c1936f71d522373dd4d13cf37bd76229595a2ba2384346"}
Jul 17 09:24:34.678407 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:34.678380    2575 generic.go:356] "Generic (PLEG): container finished" podID="718156a7-f6a4-4e41-8b63-56c317205082" containerID="48b2e98b8402bc7f51c1936f71d522373dd4d13cf37bd76229595a2ba2384346" exitCode=0
Jul 17 09:24:34.678742 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:34.678423    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm" event={"ID":"718156a7-f6a4-4e41-8b63-56c317205082","Type":"ContainerDied","Data":"48b2e98b8402bc7f51c1936f71d522373dd4d13cf37bd76229595a2ba2384346"}
Jul 17 09:24:35.690282 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:35.690244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm" event={"ID":"718156a7-f6a4-4e41-8b63-56c317205082","Type":"ContainerStarted","Data":"c9720523f69b4d6f30fdacde0f63927edc627b915fba18472a5daf92cfc65861"}
Jul 17 09:24:35.690706 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:35.690418    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:35.714600 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:35.714544    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm" podStartSLOduration=2.335254638 podStartE2EDuration="6.714524449s" totalImagesPullingTime="3.329272435s" totalInitContainerRuntime="1.049997376s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:35.712820207 +0000 UTC m=+389.104812822" watchObservedRunningTime="2026-07-17 09:24:35.714524449 +0000 UTC m=+389.106517067"
Jul 17 09:24:46.698006 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:46.697976    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-rhjtm"
Jul 17 09:24:50.133682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.133641    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"]
Jul 17 09:24:50.142624 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.142601    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.146695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.146675    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 09:24:50.146823 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.146699    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 09:24:50.146823 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.146766    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 09:24:50.146936 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.146699    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-np6hb\""
Jul 17 09:24:50.146936 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.146699    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 09:24:50.156412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.156389    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"]
Jul 17 09:24:50.268980 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.268955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.269086 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.268988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gj5k\" (UniqueName: \"kubernetes.io/projected/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-kube-api-access-9gj5k\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.269086 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.269010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tmp\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.369525 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.369490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.369685 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.369542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gj5k\" (UniqueName: \"kubernetes.io/projected/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-kube-api-access-9gj5k\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.369685 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.369571    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tmp\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.369685 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:24:50.369630    2575 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 17 09:24:50.369838 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:24:50.369702    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs podName:8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a nodeName:}" failed. No retries permitted until 2026-07-17 09:24:50.869685704 +0000 UTC m=+404.261678298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs") pod "kube-auth-proxy-7df6bb5dc-qznjf" (UID: "8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a") : secret "kube-auth-proxy-tls" not found
Jul 17 09:24:50.371784 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.371763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tmp\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.384840 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.384782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gj5k\" (UniqueName: \"kubernetes.io/projected/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-kube-api-access-9gj5k\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.874160 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.874127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:50.876555 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:50.876526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-qznjf\" (UID: \"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:51.052265 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:51.052236    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"
Jul 17 09:24:51.193429 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:51.193362    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf"]
Jul 17 09:24:51.195696 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:24:51.195674    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5157c3_9d5f_4230_a0ae_fbd57b2c7e4a.slice/crio-f5852ca4ed32f72368281f1ebe529c607d4c66b332b38f70771c5dc280d3a1f0 WatchSource:0}: Error finding container f5852ca4ed32f72368281f1ebe529c607d4c66b332b38f70771c5dc280d3a1f0: Status 404 returned error can't find the container with id f5852ca4ed32f72368281f1ebe529c607d4c66b332b38f70771c5dc280d3a1f0
Jul 17 09:24:51.766216 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:51.766174    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf" event={"ID":"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a","Type":"ContainerStarted","Data":"f5852ca4ed32f72368281f1ebe529c607d4c66b332b38f70771c5dc280d3a1f0"}
Jul 17 09:24:54.781888 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:54.781841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf" event={"ID":"8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a","Type":"ContainerStarted","Data":"c0521c62c7301f4360462c6db33701f485e032544d4b7bb3fda0bd611bc04523"}
Jul 17 09:24:54.804768 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:54.804721    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-qznjf" podStartSLOduration=2.038428836 podStartE2EDuration="4.804705609s" totalImagesPullingTime="2.766276773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:54.803030381 +0000 UTC m=+408.195022999" watchObservedRunningTime="2026-07-17 09:24:54.804705609 +0000 UTC m=+408.196698226"
Jul 17 09:24:56.325734 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.325705    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"]
Jul 17 09:24:56.329026 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.329003    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.335754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.335735    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 09:24:56.339842 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.339822    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fz9hd\""
Jul 17 09:24:56.351750 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.351728    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"]
Jul 17 09:24:56.421725 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421703    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce89b7b3-a842-4f44-894f-203708d84729-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421837 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421734    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421837 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421837 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6dx4\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-kube-api-access-t6dx4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.421944 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421901    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce89b7b3-a842-4f44-894f-203708d84729-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.422051 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.421972    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.422051 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.422007    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.522988    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523035    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce89b7b3-a842-4f44-894f-203708d84729-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523054    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523249 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523249 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523234    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523351 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523272    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523351 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523335    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6dx4\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-kube-api-access-t6dx4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce89b7b3-a842-4f44-894f-203708d84729-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523405    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523425    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523550    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523908 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce89b7b3-a842-4f44-894f-203708d84729-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.523908 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.523829    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.525756 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.525731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce89b7b3-a842-4f44-894f-203708d84729-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.526106 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.526084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce89b7b3-a842-4f44-894f-203708d84729-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.545696 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.545675    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.547288 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.547264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6dx4\" (UniqueName: \"kubernetes.io/projected/ce89b7b3-a842-4f44-894f-203708d84729-kube-api-access-t6dx4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45pw68\" (UID: \"ce89b7b3-a842-4f44-894f-203708d84729\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.639401 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.639340    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:24:56.804140 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:56.804107    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"]
Jul 17 09:24:56.805216 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:24:56.805190    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce89b7b3_a842_4f44_894f_203708d84729.slice/crio-056ab4d20d249a0e57aba13c361305e89ad86780e6bcc160b7196873017f2380 WatchSource:0}: Error finding container 056ab4d20d249a0e57aba13c361305e89ad86780e6bcc160b7196873017f2380: Status 404 returned error can't find the container with id 056ab4d20d249a0e57aba13c361305e89ad86780e6bcc160b7196873017f2380
Jul 17 09:24:57.360678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.360647    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"]
Jul 17 09:24:57.365204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.365187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.369757 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.369735    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tfrnw\""
Jul 17 09:24:57.369863 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.369780    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 09:24:57.369912 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.369901    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 09:24:57.369954 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.369944    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 09:24:57.383148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.383124    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"]
Jul 17 09:24:57.532370 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.532335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-metrics-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.532528 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.532393    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.532528 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.532431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dntkp\" (UniqueName: \"kubernetes.io/projected/a63cdf7c-8b23-46a2-82f7-d930df916630-kube-api-access-dntkp\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.532528 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.532493    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a63cdf7c-8b23-46a2-82f7-d930df916630-manager-config\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.633076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.633013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-metrics-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.633076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.633049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.633076 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.633070    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dntkp\" (UniqueName: \"kubernetes.io/projected/a63cdf7c-8b23-46a2-82f7-d930df916630-kube-api-access-dntkp\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.633267 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.633106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a63cdf7c-8b23-46a2-82f7-d930df916630-manager-config\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.633837 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.633815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a63cdf7c-8b23-46a2-82f7-d930df916630-manager-config\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.635404 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.635382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.635483 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.635469    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a63cdf7c-8b23-46a2-82f7-d930df916630-metrics-cert\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.672014 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.671986    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dntkp\" (UniqueName: \"kubernetes.io/projected/a63cdf7c-8b23-46a2-82f7-d930df916630-kube-api-access-dntkp\") pod \"lws-controller-manager-7dd756d967-fzkpn\" (UID: \"a63cdf7c-8b23-46a2-82f7-d930df916630\") " pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.674769 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.674749    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:24:57.796207 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.796177    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68" event={"ID":"ce89b7b3-a842-4f44-894f-203708d84729","Type":"ContainerStarted","Data":"056ab4d20d249a0e57aba13c361305e89ad86780e6bcc160b7196873017f2380"}
Jul 17 09:24:57.819221 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:57.819200    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"]
Jul 17 09:24:57.821320 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:24:57.821274    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63cdf7c_8b23_46a2_82f7_d930df916630.slice/crio-7374983acccd85e6591f2eea3d0257c682efd295a69753c0f42b4823afc2156c WatchSource:0}: Error finding container 7374983acccd85e6591f2eea3d0257c682efd295a69753c0f42b4823afc2156c: Status 404 returned error can't find the container with id 7374983acccd85e6591f2eea3d0257c682efd295a69753c0f42b4823afc2156c
Jul 17 09:24:58.801874 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:58.801844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn" event={"ID":"a63cdf7c-8b23-46a2-82f7-d930df916630","Type":"ContainerStarted","Data":"7374983acccd85e6591f2eea3d0257c682efd295a69753c0f42b4823afc2156c"}
Jul 17 09:24:59.807373 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:59.807338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn" event={"ID":"a63cdf7c-8b23-46a2-82f7-d930df916630","Type":"ContainerStarted","Data":"bb38e46f1bfccb5f0c3027790428478d76d6ec7b4e5a42f9f01b8bf7032ac884"}
Jul 17 09:24:59.807716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:24:59.807469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:25:10.814020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:10.813982    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn"
Jul 17 09:25:10.847760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:10.847714    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dd756d967-fzkpn" podStartSLOduration=11.998950733000001 podStartE2EDuration="13.847700268s" totalImagesPullingTime="1.848749535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:59.853868856 +0000 UTC m=+413.245861472" watchObservedRunningTime="2026-07-17 09:25:10.847700268 +0000 UTC m=+424.239692863"
Jul 17 09:25:26.279034 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:26.278988    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:25:26.279346 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:26.279072    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:25:26.279346 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:26.279105    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:25:26.924897 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:26.924863    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68" event={"ID":"ce89b7b3-a842-4f44-894f-203708d84729","Type":"ContainerStarted","Data":"a38425318724c0119d725cc6fd1547d59e9f517e1b8dbb07ffd4fb53f04ffafb"}
Jul 17 09:25:26.952639 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:26.952591    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68" podStartSLOduration=1.481101025 podStartE2EDuration="30.952574962s" totalImagesPullingTime="29.471473937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:25:26.950277871 +0000 UTC m=+440.342270492" watchObservedRunningTime="2026-07-17 09:25:26.952574962 +0000 UTC m=+440.344567578"
Jul 17 09:25:27.640080 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:27.640045    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:25:27.644714 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:27.644689    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:25:27.928894 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:27.928816    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:25:27.929824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:25:27.929807    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45pw68"
Jul 17 09:26:36.594507 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.594470    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"]
Jul 17 09:26:36.597848 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.597829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.604139 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.603911    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 09:26:36.604139 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.604048    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 09:26:36.604268 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.604253    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 09:26:36.604669 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.604653    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 09:26:36.605283 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.605268    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xwls4\""
Jul 17 09:26:36.622240 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.622211    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"]
Jul 17 09:26:36.763034 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.763000    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3de5614f-4875-48eb-b5aa-381bd6d167af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.763193 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.763064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3de5614f-4875-48eb-b5aa-381bd6d167af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.763193 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.763124    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rjh\" (UniqueName: \"kubernetes.io/projected/3de5614f-4875-48eb-b5aa-381bd6d167af-kube-api-access-27rjh\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.864049 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.863979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3de5614f-4875-48eb-b5aa-381bd6d167af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.864049 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.864034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3de5614f-4875-48eb-b5aa-381bd6d167af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.864245 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.864064    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rjh\" (UniqueName: \"kubernetes.io/projected/3de5614f-4875-48eb-b5aa-381bd6d167af-kube-api-access-27rjh\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.864739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.864715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3de5614f-4875-48eb-b5aa-381bd6d167af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.866514 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.866491    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3de5614f-4875-48eb-b5aa-381bd6d167af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.877573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.877544    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rjh\" (UniqueName: \"kubernetes.io/projected/3de5614f-4875-48eb-b5aa-381bd6d167af-kube-api-access-27rjh\") pod \"kuadrant-console-plugin-66b5774985-c4ntr\" (UID: \"3de5614f-4875-48eb-b5aa-381bd6d167af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:36.907390 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:36.907364    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"
Jul 17 09:26:37.044933 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:37.044850    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr"]
Jul 17 09:26:37.047176 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:26:37.047146    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3de5614f_4875_48eb_b5aa_381bd6d167af.slice/crio-5f27769e83c2cb7d3f2e5e761801b4b196b87e28c4e256f6441fedaeb7501dcf WatchSource:0}: Error finding container 5f27769e83c2cb7d3f2e5e761801b4b196b87e28c4e256f6441fedaeb7501dcf: Status 404 returned error can't find the container with id 5f27769e83c2cb7d3f2e5e761801b4b196b87e28c4e256f6441fedaeb7501dcf
Jul 17 09:26:37.239692 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:37.239656    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr" event={"ID":"3de5614f-4875-48eb-b5aa-381bd6d167af","Type":"ContainerStarted","Data":"5f27769e83c2cb7d3f2e5e761801b4b196b87e28c4e256f6441fedaeb7501dcf"}
Jul 17 09:26:59.356333 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:59.356272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr" event={"ID":"3de5614f-4875-48eb-b5aa-381bd6d167af","Type":"ContainerStarted","Data":"738dffd83ad63f53363ac1f1a72dfb774ab9d9254aa5496571973a09b5c04fa8"}
Jul 17 09:26:59.379498 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:26:59.379442    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c4ntr" podStartSLOduration=1.681844322 podStartE2EDuration="23.379427861s" totalImagesPullingTime="21.697583539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:26:59.378575547 +0000 UTC m=+532.770568165" watchObservedRunningTime="2026-07-17 09:26:59.379427861 +0000 UTC m=+532.771420477"
Jul 17 09:27:06.183808 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:06.183776    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:27:21.103885 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.103848    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:21.263437 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.263404    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:21.263595 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.263516    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:21.266472 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.266445    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b89mc\""
Jul 17 09:27:21.363120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.363058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9psq\" (UniqueName: \"kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq\") pod \"authorino-58cd955675-z4djf\" (UID: \"8721bf1c-2af6-4327-a8c1-5a48939da6ca\") " pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:21.463660 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.463629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9psq\" (UniqueName: \"kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq\") pod \"authorino-58cd955675-z4djf\" (UID: \"8721bf1c-2af6-4327-a8c1-5a48939da6ca\") " pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:21.473185 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.473160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9psq\" (UniqueName: \"kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq\") pod \"authorino-58cd955675-z4djf\" (UID: \"8721bf1c-2af6-4327-a8c1-5a48939da6ca\") " pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:21.572844 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.572813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:21.790978 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:21.790951    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:21.793002 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:27:21.792975    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8721bf1c_2af6_4327_a8c1_5a48939da6ca.slice/crio-170083394a1a6819d39760ed3d44f70a57d7d10a482312bc313953fea2172aeb WatchSource:0}: Error finding container 170083394a1a6819d39760ed3d44f70a57d7d10a482312bc313953fea2172aeb: Status 404 returned error can't find the container with id 170083394a1a6819d39760ed3d44f70a57d7d10a482312bc313953fea2172aeb
Jul 17 09:27:22.458965 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:22.458931    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z4djf" event={"ID":"8721bf1c-2af6-4327-a8c1-5a48939da6ca","Type":"ContainerStarted","Data":"170083394a1a6819d39760ed3d44f70a57d7d10a482312bc313953fea2172aeb"}
Jul 17 09:27:26.265991 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:26.265951    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:26.481504 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:26.481461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z4djf" event={"ID":"8721bf1c-2af6-4327-a8c1-5a48939da6ca","Type":"ContainerStarted","Data":"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"}
Jul 17 09:27:26.500992 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:26.500945    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-z4djf" podStartSLOduration=1.384457412 podStartE2EDuration="5.500931239s" totalImagesPullingTime="4.116473827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:26.498417819 +0000 UTC m=+559.890410437" watchObservedRunningTime="2026-07-17 09:27:26.500931239 +0000 UTC m=+559.892923856"
Jul 17 09:27:27.485907 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:27.485865    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-z4djf" podUID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" containerName="authorino" containerID="cri-o://c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603" gracePeriod=30
Jul 17 09:27:27.764101 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:27.764076    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:27.822450 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:27.822417    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9psq\" (UniqueName: \"kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq\") pod \"8721bf1c-2af6-4327-a8c1-5a48939da6ca\" (UID: \"8721bf1c-2af6-4327-a8c1-5a48939da6ca\") "
Jul 17 09:27:27.824611 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:27.824588    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq" (OuterVolumeSpecName: "kube-api-access-l9psq") pod "8721bf1c-2af6-4327-a8c1-5a48939da6ca" (UID: "8721bf1c-2af6-4327-a8c1-5a48939da6ca"). InnerVolumeSpecName "kube-api-access-l9psq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:27:27.923126 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:27.923098    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9psq\" (UniqueName: \"kubernetes.io/projected/8721bf1c-2af6-4327-a8c1-5a48939da6ca-kube-api-access-l9psq\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:28.491058 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.491022    2575 generic.go:356] "Generic (PLEG): container finished" podID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" containerID="c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603" exitCode=0
Jul 17 09:27:28.491543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.491063    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-z4djf"
Jul 17 09:27:28.491543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.491087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z4djf" event={"ID":"8721bf1c-2af6-4327-a8c1-5a48939da6ca","Type":"ContainerDied","Data":"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"}
Jul 17 09:27:28.491543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.491117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-z4djf" event={"ID":"8721bf1c-2af6-4327-a8c1-5a48939da6ca","Type":"ContainerDied","Data":"170083394a1a6819d39760ed3d44f70a57d7d10a482312bc313953fea2172aeb"}
Jul 17 09:27:28.491543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.491135    2575 scope.go:117] "RemoveContainer" containerID="c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"
Jul 17 09:27:28.499618 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.499594    2575 scope.go:117] "RemoveContainer" containerID="c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"
Jul 17 09:27:28.499854 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:27:28.499837    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603\": container with ID starting with c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603 not found: ID does not exist" containerID="c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"
Jul 17 09:27:28.499904 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.499863    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603"} err="failed to get container status \"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603\": rpc error: code = NotFound desc = could not find container \"c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603\": container with ID starting with c25794bc800fd4337f1f7bfe9ee4712ed12c286c3bc38f04e8f1110edf3ed603 not found: ID does not exist"
Jul 17 09:27:28.512381 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.512354    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:28.516257 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:28.516234    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-z4djf"]
Jul 17 09:27:29.153227 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:29.153196    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" path="/var/lib/kubelet/pods/8721bf1c-2af6-4327-a8c1-5a48939da6ca/volumes"
Jul 17 09:27:31.204384 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.204337    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85d4cd9497-lr2sn" podUID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" containerName="console" containerID="cri-o://865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4" gracePeriod=15
Jul 17 09:27:31.454764 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.454705    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:27:31.506681 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.506652    2575 generic.go:356] "Generic (PLEG): container finished" podID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" containerID="865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4" exitCode=0
Jul 17 09:27:31.506821 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.506691    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4cd9497-lr2sn" event={"ID":"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3","Type":"ContainerDied","Data":"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"}
Jul 17 09:27:31.506821 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.506711    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d4cd9497-lr2sn"
Jul 17 09:27:31.506821 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.506733    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d4cd9497-lr2sn" event={"ID":"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3","Type":"ContainerDied","Data":"510d50aff3e51e8381df852cf7eb5d617a52951483ff21fa5b8f70a3b48c47e1"}
Jul 17 09:27:31.506821 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.506754    2575 scope.go:117] "RemoveContainer" containerID="865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"
Jul 17 09:27:31.514885 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.514866    2575 scope.go:117] "RemoveContainer" containerID="865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"
Jul 17 09:27:31.515146 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:27:31.515128    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4\": container with ID starting with 865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4 not found: ID does not exist" containerID="865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"
Jul 17 09:27:31.515191 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.515155    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4"} err="failed to get container status \"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4\": rpc error: code = NotFound desc = could not find container \"865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4\": container with ID starting with 865973f26c2732ddb2e0456a025024829006981096581d4f71289f19e5822aa4 not found: ID does not exist"
Jul 17 09:27:31.547074 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547047    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547178 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547076    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547178 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547101    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547178 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547136    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547178 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547152    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqp8\" (UniqueName: \"kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547181    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.547553 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547520    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:27:31.547662 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547608    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:27:31.547662 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.547643    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config" (OuterVolumeSpecName: "console-config") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:27:31.549268 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.549248    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8" (OuterVolumeSpecName: "kube-api-access-pgqp8") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "kube-api-access-pgqp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:27:31.549650 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.549631    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:27:31.549719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.549700    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:27:31.647889 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.647856    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle\") pod \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\" (UID: \"dc979de2-4f08-45ec-a00f-dd6ceab4bdf3\") "
Jul 17 09:27:31.648074 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648052    2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-oauth-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648074 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648063    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqp8\" (UniqueName: \"kubernetes.io/projected/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-kube-api-access-pgqp8\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648076    2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-config\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648084    2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-oauth-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648092    2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-console-serving-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648194 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648102    2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.648417 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.648282    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" (UID: "dc979de2-4f08-45ec-a00f-dd6ceab4bdf3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:27:31.748839 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.748809    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3-trusted-ca-bundle\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:31.833990 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.833964    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:27:31.841937 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:31.841915    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d4cd9497-lr2sn"]
Jul 17 09:27:33.152880 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:33.152843    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" path="/var/lib/kubelet/pods/dc979de2-4f08-45ec-a00f-dd6ceab4bdf3/volumes"
Jul 17 09:27:48.994818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.994784    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995103    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" containerName="console"
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995113    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" containerName="console"
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995126    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" containerName="authorino"
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995133    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" containerName="authorino"
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995193    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc979de2-4f08-45ec-a00f-dd6ceab4bdf3" containerName="console"
Jul 17 09:27:48.995480 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.995204    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8721bf1c-2af6-4327-a8c1-5a48939da6ca" containerName="authorino"
Jul 17 09:27:48.999503 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:48.999486    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:49.003941 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.003923    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-b89mc\""
Jul 17 09:27:49.016751 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.016729    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:49.076478 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.076443    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7spk\" (UniqueName: \"kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk\") pod \"authorino-5d4b67c446-j64m9\" (UID: \"76fe4654-2c2e-441a-9a43-6a977835243e\") " pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:49.177109 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.177073    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7spk\" (UniqueName: \"kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk\") pod \"authorino-5d4b67c446-j64m9\" (UID: \"76fe4654-2c2e-441a-9a43-6a977835243e\") " pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:49.195043 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.195017    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7spk\" (UniqueName: \"kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk\") pod \"authorino-5d4b67c446-j64m9\" (UID: \"76fe4654-2c2e-441a-9a43-6a977835243e\") " pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:49.244388 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.244361    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:49.244607 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.244595    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:49.300287 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.300257    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:49.304763 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.304740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:49.318217 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.318188    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:49.378754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.378723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqq2v\" (UniqueName: \"kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v\") pod \"authorino-686fd7d897-vzmx9\" (UID: \"3f24c8ff-28ec-45b7-af15-63d21d1aaec0\") " pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:49.392979 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.392959    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:49.395445 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:27:49.395415    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76fe4654_2c2e_441a_9a43_6a977835243e.slice/crio-b5cd90dd21b4e4b575c401d5535ec121d600471a779580ec73eb667d8c2631bb WatchSource:0}: Error finding container b5cd90dd21b4e4b575c401d5535ec121d600471a779580ec73eb667d8c2631bb: Status 404 returned error can't find the container with id b5cd90dd21b4e4b575c401d5535ec121d600471a779580ec73eb667d8c2631bb
Jul 17 09:27:49.479360 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.479333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqq2v\" (UniqueName: \"kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v\") pod \"authorino-686fd7d897-vzmx9\" (UID: \"3f24c8ff-28ec-45b7-af15-63d21d1aaec0\") " pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:49.498010 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.497948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqq2v\" (UniqueName: \"kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v\") pod \"authorino-686fd7d897-vzmx9\" (UID: \"3f24c8ff-28ec-45b7-af15-63d21d1aaec0\") " pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:49.559557 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.559528    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:49.559769 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.559757    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:49.595851 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.595822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j64m9" event={"ID":"76fe4654-2c2e-441a-9a43-6a977835243e","Type":"ContainerStarted","Data":"b5cd90dd21b4e4b575c401d5535ec121d600471a779580ec73eb667d8c2631bb"}
Jul 17 09:27:49.618997 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.618874    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:27:49.622545 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.622524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.627047 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.627027    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 09:27:49.645946 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.645911    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:27:49.681589 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.681563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzgv\" (UniqueName: \"kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.681702 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.681600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.709431 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.709401    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:49.710429 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:27:49.710409    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f24c8ff_28ec_45b7_af15_63d21d1aaec0.slice/crio-18cb245a0e020578be284ef75b8e70d84b710cc6c701ad008fed187c71c9cf8c WatchSource:0}: Error finding container 18cb245a0e020578be284ef75b8e70d84b710cc6c701ad008fed187c71c9cf8c: Status 404 returned error can't find the container with id 18cb245a0e020578be284ef75b8e70d84b710cc6c701ad008fed187c71c9cf8c
Jul 17 09:27:49.781973 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.781952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzgv\" (UniqueName: \"kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.782064 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.781983    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.784078 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.784060    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.794438 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.794415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzgv\" (UniqueName: \"kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv\") pod \"authorino-5dc898b6c7-phxx5\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") " pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:49.932037 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:49.932005    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:27:50.069682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.069658    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:27:50.082358 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:27:50.082332    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c6d6cba_eb1f_42e8_a117_e4391514a00c.slice/crio-fb94296be09f2a84a97cd8500b20696ce51786ee234a6e70eda29a58b302dc6a WatchSource:0}: Error finding container fb94296be09f2a84a97cd8500b20696ce51786ee234a6e70eda29a58b302dc6a: Status 404 returned error can't find the container with id fb94296be09f2a84a97cd8500b20696ce51786ee234a6e70eda29a58b302dc6a
Jul 17 09:27:50.604073 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.604038    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-686fd7d897-vzmx9" event={"ID":"3f24c8ff-28ec-45b7-af15-63d21d1aaec0","Type":"ContainerStarted","Data":"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"}
Jul 17 09:27:50.604073 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.604077    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-686fd7d897-vzmx9" podUID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" containerName="authorino" containerID="cri-o://e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160" gracePeriod=30
Jul 17 09:27:50.604359 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.604087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-686fd7d897-vzmx9" event={"ID":"3f24c8ff-28ec-45b7-af15-63d21d1aaec0","Type":"ContainerStarted","Data":"18cb245a0e020578be284ef75b8e70d84b710cc6c701ad008fed187c71c9cf8c"}
Jul 17 09:27:50.605780 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.605756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j64m9" event={"ID":"76fe4654-2c2e-441a-9a43-6a977835243e","Type":"ContainerStarted","Data":"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"}
Jul 17 09:27:50.605926 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.605799    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-j64m9" podUID="76fe4654-2c2e-441a-9a43-6a977835243e" containerName="authorino" containerID="cri-o://836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d" gracePeriod=30
Jul 17 09:27:50.607608 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.607580    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" event={"ID":"7c6d6cba-eb1f-42e8-a117-e4391514a00c","Type":"ContainerStarted","Data":"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"}
Jul 17 09:27:50.607705 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.607615    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" event={"ID":"7c6d6cba-eb1f-42e8-a117-e4391514a00c","Type":"ContainerStarted","Data":"fb94296be09f2a84a97cd8500b20696ce51786ee234a6e70eda29a58b302dc6a"}
Jul 17 09:27:50.622764 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.622698    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-686fd7d897-vzmx9" podStartSLOduration=1.28769848 podStartE2EDuration="1.622686617s" totalImagesPullingTime="334.988137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:50.621726843 +0000 UTC m=+584.013719462" watchObservedRunningTime="2026-07-17 09:27:50.622686617 +0000 UTC m=+584.014679288"
Jul 17 09:27:50.639031 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.638998    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-j64m9" podStartSLOduration=2.276192254 podStartE2EDuration="2.638984905s" totalImagesPullingTime="362.792651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:50.638799712 +0000 UTC m=+584.030792342" watchObservedRunningTime="2026-07-17 09:27:50.638984905 +0000 UTC m=+584.030977521"
Jul 17 09:27:50.655950 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.655908    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" podStartSLOduration=1.2855378929999999 podStartE2EDuration="1.655896265s" totalImagesPullingTime="370.358372ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:50.65560732 +0000 UTC m=+584.047599939" watchObservedRunningTime="2026-07-17 09:27:50.655896265 +0000 UTC m=+584.047888883"
Jul 17 09:27:50.845859 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.845832    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:50.865488 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.865464    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:50.891070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.891046    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7spk\" (UniqueName: \"kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk\") pod \"76fe4654-2c2e-441a-9a43-6a977835243e\" (UID: \"76fe4654-2c2e-441a-9a43-6a977835243e\") "
Jul 17 09:27:50.891212 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.891098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqq2v\" (UniqueName: \"kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v\") pod \"3f24c8ff-28ec-45b7-af15-63d21d1aaec0\" (UID: \"3f24c8ff-28ec-45b7-af15-63d21d1aaec0\") "
Jul 17 09:27:50.893147 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.893113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v" (OuterVolumeSpecName: "kube-api-access-jqq2v") pod "3f24c8ff-28ec-45b7-af15-63d21d1aaec0" (UID: "3f24c8ff-28ec-45b7-af15-63d21d1aaec0"). InnerVolumeSpecName "kube-api-access-jqq2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:27:50.893241 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.893156    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk" (OuterVolumeSpecName: "kube-api-access-w7spk") pod "76fe4654-2c2e-441a-9a43-6a977835243e" (UID: "76fe4654-2c2e-441a-9a43-6a977835243e"). InnerVolumeSpecName "kube-api-access-w7spk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:27:50.992672 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.992645    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7spk\" (UniqueName: \"kubernetes.io/projected/76fe4654-2c2e-441a-9a43-6a977835243e-kube-api-access-w7spk\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:50.992672 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:50.992669    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqq2v\" (UniqueName: \"kubernetes.io/projected/3f24c8ff-28ec-45b7-af15-63d21d1aaec0-kube-api-access-jqq2v\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:27:51.613319 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.613267    2575 generic.go:356] "Generic (PLEG): container finished" podID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" containerID="e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160" exitCode=0
Jul 17 09:27:51.613793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.613335    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-686fd7d897-vzmx9"
Jul 17 09:27:51.613793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.613354    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-686fd7d897-vzmx9" event={"ID":"3f24c8ff-28ec-45b7-af15-63d21d1aaec0","Type":"ContainerDied","Data":"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"}
Jul 17 09:27:51.613793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.613398    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-686fd7d897-vzmx9" event={"ID":"3f24c8ff-28ec-45b7-af15-63d21d1aaec0","Type":"ContainerDied","Data":"18cb245a0e020578be284ef75b8e70d84b710cc6c701ad008fed187c71c9cf8c"}
Jul 17 09:27:51.613793 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.613428    2575 scope.go:117] "RemoveContainer" containerID="e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"
Jul 17 09:27:51.614754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.614725    2575 generic.go:356] "Generic (PLEG): container finished" podID="76fe4654-2c2e-441a-9a43-6a977835243e" containerID="836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d" exitCode=0
Jul 17 09:27:51.614870 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.614776    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j64m9"
Jul 17 09:27:51.614870 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.614816    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j64m9" event={"ID":"76fe4654-2c2e-441a-9a43-6a977835243e","Type":"ContainerDied","Data":"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"}
Jul 17 09:27:51.614870 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.614857    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j64m9" event={"ID":"76fe4654-2c2e-441a-9a43-6a977835243e","Type":"ContainerDied","Data":"b5cd90dd21b4e4b575c401d5535ec121d600471a779580ec73eb667d8c2631bb"}
Jul 17 09:27:51.621813 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.621798    2575 scope.go:117] "RemoveContainer" containerID="e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"
Jul 17 09:27:51.622034 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:27:51.622019    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160\": container with ID starting with e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160 not found: ID does not exist" containerID="e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"
Jul 17 09:27:51.622089 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.622041    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160"} err="failed to get container status \"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160\": rpc error: code = NotFound desc = could not find container \"e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160\": container with ID starting with e2e70c732f91faea94f187d8c8baf32ebc5ea9261212fc0d2142ae2e92f83160 not found: ID does not exist"
Jul 17 09:27:51.622089 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.622055    2575 scope.go:117] "RemoveContainer" containerID="836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"
Jul 17 09:27:51.629567 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.629552    2575 scope.go:117] "RemoveContainer" containerID="836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"
Jul 17 09:27:51.629830 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:27:51.629811    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d\": container with ID starting with 836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d not found: ID does not exist" containerID="836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"
Jul 17 09:27:51.629903 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.629834    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d"} err="failed to get container status \"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d\": rpc error: code = NotFound desc = could not find container \"836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d\": container with ID starting with 836cba1ec7d6fe9006ea8eb1f685202362805bf25c4f4bfd1e20aac2357f946d not found: ID does not exist"
Jul 17 09:27:51.631369 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.631350    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:51.635773 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.635751    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j64m9"]
Jul 17 09:27:51.647414 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.647393    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:51.650985 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:51.650966    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-686fd7d897-vzmx9"]
Jul 17 09:27:53.153212 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:53.153175    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" path="/var/lib/kubelet/pods/3f24c8ff-28ec-45b7-af15-63d21d1aaec0/volumes"
Jul 17 09:27:53.153727 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:27:53.153690    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76fe4654-2c2e-441a-9a43-6a977835243e" path="/var/lib/kubelet/pods/76fe4654-2c2e-441a-9a43-6a977835243e/volumes"
Jul 17 09:28:07.078355 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:28:07.078325    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:28:07.083974 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:28:07.083946    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:28:07.086034 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:28:07.086007    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:28:07.092169 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:28:07.092153    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:29:17.820357 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820323    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"]
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820651    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76fe4654-2c2e-441a-9a43-6a977835243e" containerName="authorino"
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820662    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="76fe4654-2c2e-441a-9a43-6a977835243e" containerName="authorino"
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820670    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" containerName="authorino"
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820675    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" containerName="authorino"
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820744    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f24c8ff-28ec-45b7-af15-63d21d1aaec0" containerName="authorino"
Jul 17 09:29:17.822678 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.820757    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="76fe4654-2c2e-441a-9a43-6a977835243e" containerName="authorino"
Jul 17 09:29:17.823582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.823564    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:17.826705 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.826683    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mj6v8\""
Jul 17 09:29:17.826705 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.826701    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 09:29:17.827719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.827692    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 09:29:17.827719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.827710    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 09:29:17.859220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:17.859192    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"]
Jul 17 09:29:18.017647 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017619    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.017826 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmv2b\" (UniqueName: \"kubernetes.io/projected/9f1cf106-17a4-4882-9157-e66acf54a339-kube-api-access-cmv2b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.017826 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017732    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.017826 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.017990 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f1cf106-17a4-4882-9157-e66acf54a339-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.017990 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.017889    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.118803 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.118803 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.118803 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmv2b\" (UniqueName: \"kubernetes.io/projected/9f1cf106-17a4-4882-9157-e66acf54a339-kube-api-access-cmv2b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.118881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f1cf106-17a4-4882-9157-e66acf54a339-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119192 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.119141    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119243 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.119187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.119289 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.119232    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.121122 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.121098    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f1cf106-17a4-4882-9157-e66acf54a339-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.121363 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.121347    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f1cf106-17a4-4882-9157-e66acf54a339-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.127400 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.127379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmv2b\" (UniqueName: \"kubernetes.io/projected/9f1cf106-17a4-4882-9157-e66acf54a339-kube-api-access-cmv2b\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ls46t\" (UID: \"9f1cf106-17a4-4882-9157-e66acf54a339\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.133257 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.133234    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:18.269690 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.269669    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"]
Jul 17 09:29:18.271743 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:29:18.271716    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1cf106_17a4_4882_9157_e66acf54a339.slice/crio-d49a77d2c9105f4b37ae3d8c34eef738e08eec4107071e10f6d1070a99256e22 WatchSource:0}: Error finding container d49a77d2c9105f4b37ae3d8c34eef738e08eec4107071e10f6d1070a99256e22: Status 404 returned error can't find the container with id d49a77d2c9105f4b37ae3d8c34eef738e08eec4107071e10f6d1070a99256e22
Jul 17 09:29:18.273056 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:18.273035    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:29:19.009262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:19.009219    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t" event={"ID":"9f1cf106-17a4-4882-9157-e66acf54a339","Type":"ContainerStarted","Data":"d49a77d2c9105f4b37ae3d8c34eef738e08eec4107071e10f6d1070a99256e22"}
Jul 17 09:29:20.016844 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:20.016809    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t" event={"ID":"9f1cf106-17a4-4882-9157-e66acf54a339","Type":"ContainerStarted","Data":"cf5c3912cb714730d69ab88f53fc70aefe02caeae386ad2a11df797f12113aa0"}
Jul 17 09:29:23.496631 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.496573    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t" podStartSLOduration=5.288039926 podStartE2EDuration="6.496556153s" totalImagesPullingTime="1.208516227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:29:20.042385523 +0000 UTC m=+673.434378141" watchObservedRunningTime="2026-07-17 09:29:23.496556153 +0000 UTC m=+676.888548771"
Jul 17 09:29:23.497310 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.497265    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"]
Jul 17 09:29:23.500889 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.500871    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.503923 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.503903    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 17 09:29:23.512345 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.512322    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"]
Jul 17 09:29:23.564273 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564250    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.564421 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564285    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5d0c117-28f7-4646-8131-2784d20132fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.564421 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564335    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4c6\" (UniqueName: \"kubernetes.io/projected/f5d0c117-28f7-4646-8131-2784d20132fa-kube-api-access-sm4c6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.564421 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.564523 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.564556 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.564524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665595 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665719    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5d0c117-28f7-4646-8131-2784d20132fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.665754 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.665737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4c6\" (UniqueName: \"kubernetes.io/projected/f5d0c117-28f7-4646-8131-2784d20132fa-kube-api-access-sm4c6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.666056 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.666030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.666121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.666065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.666121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.666079    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.668209 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.668182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5d0c117-28f7-4646-8131-2784d20132fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.668371 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.668245    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5d0c117-28f7-4646-8131-2784d20132fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.678212 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.678186    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4c6\" (UniqueName: \"kubernetes.io/projected/f5d0c117-28f7-4646-8131-2784d20132fa-kube-api-access-sm4c6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk\" (UID: \"f5d0c117-28f7-4646-8131-2784d20132fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.812043 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.811968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:23.941957 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:23.941929    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"]
Jul 17 09:29:23.943449 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:29:23.943423    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d0c117_28f7_4646_8131_2784d20132fa.slice/crio-f3a4bbfbfae1e54742b952bed44f97f7267410d281a7aabf4870dddce3e6f883 WatchSource:0}: Error finding container f3a4bbfbfae1e54742b952bed44f97f7267410d281a7aabf4870dddce3e6f883: Status 404 returned error can't find the container with id f3a4bbfbfae1e54742b952bed44f97f7267410d281a7aabf4870dddce3e6f883
Jul 17 09:29:24.041104 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:24.041071    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk" event={"ID":"f5d0c117-28f7-4646-8131-2784d20132fa","Type":"ContainerStarted","Data":"f3a4bbfbfae1e54742b952bed44f97f7267410d281a7aabf4870dddce3e6f883"}
Jul 17 09:29:25.047334 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:25.047272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk" event={"ID":"f5d0c117-28f7-4646-8131-2784d20132fa","Type":"ContainerStarted","Data":"3bbce84e763e7d36d63797e98a9976bbf3bb615be0223e140830e7a638059257"}
Jul 17 09:29:25.070617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:25.070572    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk" podStartSLOduration=1.7409485230000001 podStartE2EDuration="2.070556727s" totalImagesPullingTime="329.608204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:29:25.068765451 +0000 UTC m=+678.460758069" watchObservedRunningTime="2026-07-17 09:29:25.070556727 +0000 UTC m=+678.462549345"
Jul 17 09:29:28.133515 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:28.133485    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:28.133906 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:28.133542    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:28.145732 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:28.145704    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:29.077598 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:29.077568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ls46t"
Jul 17 09:29:33.812710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:33.812678    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:33.813068 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:33.812731    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:33.825176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:33.825154    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:34.100729 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:34.100639    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk"
Jul 17 09:29:58.611801 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.611763    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"]
Jul 17 09:29:58.617360 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.617343    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.620569 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.620552    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 17 09:29:58.633700 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.633676    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"]
Jul 17 09:29:58.772095 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.772223 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772129    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6j4f\" (UniqueName: \"kubernetes.io/projected/d3dc8aaf-03a1-4751-8290-8d336ceffd41-kube-api-access-j6j4f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.772223 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772171    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.772223 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.772365 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772225    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.772365 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.772260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873571 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873505    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873571 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873547    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873571 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873564    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6j4f\" (UniqueName: \"kubernetes.io/projected/d3dc8aaf-03a1-4751-8290-8d336ceffd41-kube-api-access-j6j4f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.873964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.874020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.874020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.873989    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.875940 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.875920    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3dc8aaf-03a1-4751-8290-8d336ceffd41-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.876096 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.876077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3dc8aaf-03a1-4751-8290-8d336ceffd41-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.892979 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.892956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6j4f\" (UniqueName: \"kubernetes.io/projected/d3dc8aaf-03a1-4751-8290-8d336ceffd41-kube-api-access-j6j4f\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp\" (UID: \"d3dc8aaf-03a1-4751-8290-8d336ceffd41\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:58.927626 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:58.927603    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:29:59.061961 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:59.061936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"]
Jul 17 09:29:59.064178 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:29:59.064148    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3dc8aaf_03a1_4751_8290_8d336ceffd41.slice/crio-fb117f98d396e22482a7482e837221be19ef4fbc1234e899435bcc1d6d0cb3db WatchSource:0}: Error finding container fb117f98d396e22482a7482e837221be19ef4fbc1234e899435bcc1d6d0cb3db: Status 404 returned error can't find the container with id fb117f98d396e22482a7482e837221be19ef4fbc1234e899435bcc1d6d0cb3db
Jul 17 09:29:59.209151 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:29:59.209068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp" event={"ID":"d3dc8aaf-03a1-4751-8290-8d336ceffd41","Type":"ContainerStarted","Data":"fb117f98d396e22482a7482e837221be19ef4fbc1234e899435bcc1d6d0cb3db"}
Jul 17 09:30:00.215319 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:00.215206    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp" event={"ID":"d3dc8aaf-03a1-4751-8290-8d336ceffd41","Type":"ContainerStarted","Data":"32c2f129da8f6104f7ae8c952b6a388e0893c9f2d3309a7dc918c89c41375840"}
Jul 17 09:30:08.928294 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:08.928259    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:30:08.928696 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:08.928327    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:30:08.940655 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:08.940629    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:30:08.961629 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:08.961582    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp" podStartSLOduration=10.781100168 podStartE2EDuration="10.961567798s" totalImagesPullingTime="180.46763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:30:00.242805096 +0000 UTC m=+713.634797713" watchObservedRunningTime="2026-07-17 09:30:08.961567798 +0000 UTC m=+722.353560414"
Jul 17 09:30:09.266276 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:30:09.266239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp"
Jul 17 09:31:14.233106 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.233068    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:31:14.233594 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.233261    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" podUID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" containerName="authorino" containerID="cri-o://9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582" gracePeriod=30
Jul 17 09:31:14.475733 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.475709    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:31:14.554412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.554336    2575 generic.go:356] "Generic (PLEG): container finished" podID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" containerID="9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582" exitCode=0
Jul 17 09:31:14.554412 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.554377    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc898b6c7-phxx5"
Jul 17 09:31:14.554583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.554428    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" event={"ID":"7c6d6cba-eb1f-42e8-a117-e4391514a00c","Type":"ContainerDied","Data":"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"}
Jul 17 09:31:14.554583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.554468    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dc898b6c7-phxx5" event={"ID":"7c6d6cba-eb1f-42e8-a117-e4391514a00c","Type":"ContainerDied","Data":"fb94296be09f2a84a97cd8500b20696ce51786ee234a6e70eda29a58b302dc6a"}
Jul 17 09:31:14.554583 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.554486    2575 scope.go:117] "RemoveContainer" containerID="9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"
Jul 17 09:31:14.562648 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.562633    2575 scope.go:117] "RemoveContainer" containerID="9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"
Jul 17 09:31:14.562891 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:31:14.562873    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582\": container with ID starting with 9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582 not found: ID does not exist" containerID="9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"
Jul 17 09:31:14.562952 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.562900    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582"} err="failed to get container status \"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582\": rpc error: code = NotFound desc = could not find container \"9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582\": container with ID starting with 9822968c526a9d2c5c2d0b1c2540f2078199c63ebee287ff7bcf320aa4b82582 not found: ID does not exist"
Jul 17 09:31:14.610246 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.610217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdzgv\" (UniqueName: \"kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv\") pod \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") "
Jul 17 09:31:14.610404 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.610337    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert\") pod \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\" (UID: \"7c6d6cba-eb1f-42e8-a117-e4391514a00c\") "
Jul 17 09:31:14.612246 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.612222    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv" (OuterVolumeSpecName: "kube-api-access-zdzgv") pod "7c6d6cba-eb1f-42e8-a117-e4391514a00c" (UID: "7c6d6cba-eb1f-42e8-a117-e4391514a00c"). InnerVolumeSpecName "kube-api-access-zdzgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:31:14.620214 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.620193    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7c6d6cba-eb1f-42e8-a117-e4391514a00c" (UID: "7c6d6cba-eb1f-42e8-a117-e4391514a00c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:31:14.711144 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.711108    2575 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c6d6cba-eb1f-42e8-a117-e4391514a00c-tls-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:31:14.711144 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.711141    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdzgv\" (UniqueName: \"kubernetes.io/projected/7c6d6cba-eb1f-42e8-a117-e4391514a00c-kube-api-access-zdzgv\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:31:14.882628 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.882601    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:31:14.888459 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:14.888439    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc898b6c7-phxx5"]
Jul 17 09:31:15.152522 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:31:15.152438    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" path="/var/lib/kubelet/pods/7c6d6cba-eb1f-42e8-a117-e4391514a00c/volumes"
Jul 17 09:33:07.108473 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:33:07.108395    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:33:07.113879 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:33:07.113856    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:33:07.114807 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:33:07.114789    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:33:07.120763 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:33:07.120741    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:38:07.138632 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:38:07.138603    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:38:07.143706 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:38:07.143685    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:38:07.144019 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:38:07.144001    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:38:07.149935 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:38:07.149914    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:43:07.176709 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:43:07.176685    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:43:07.178210 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:43:07.178190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:43:07.182225 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:43:07.182205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:43:07.183804 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:43:07.183786    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:45:00.150550 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.150471    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"]
Jul 17 09:45:00.150994 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.150813    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" containerName="authorino"
Jul 17 09:45:00.150994 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.150826    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" containerName="authorino"
Jul 17 09:45:00.150994 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.150899    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6d6cba-eb1f-42e8-a117-e4391514a00c" containerName="authorino"
Jul 17 09:45:00.153773 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.153755    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"
Jul 17 09:45:00.156490 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.156467    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:45:00.163645 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.163630    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"
Jul 17 09:45:00.182783 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.182758    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"]
Jul 17 09:45:00.303077 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.303049    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"]
Jul 17 09:45:00.305350 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:45:00.305322    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db63d64_322a_477e_b0e5_6e8ceef76e33.slice/crio-974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7 WatchSource:0}: Error finding container 974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7: Status 404 returned error can't find the container with id 974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7
Jul 17 09:45:00.307133 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:00.307114    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:45:01.296406 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:01.296369    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m" event={"ID":"9db63d64-322a-477e-b0e5-6e8ceef76e33","Type":"ContainerStarted","Data":"974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7"}
Jul 17 09:45:05.315745 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:05.315652    2575 generic.go:356] "Generic (PLEG): container finished" podID="9db63d64-322a-477e-b0e5-6e8ceef76e33" containerID="d157552d853bd22db444957c6112f039653cfb2fa76fb4f10335ec884d8115db" exitCode=0
Jul 17 09:45:05.316107 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:05.315736    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m" event={"ID":"9db63d64-322a-477e-b0e5-6e8ceef76e33","Type":"ContainerDied","Data":"d157552d853bd22db444957c6112f039653cfb2fa76fb4f10335ec884d8115db"}
Jul 17 09:45:06.448496 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:06.448472    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"
Jul 17 09:45:07.325906 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:07.325879    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m"
Jul 17 09:45:07.326070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:07.325906    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-rlc4m" event={"ID":"9db63d64-322a-477e-b0e5-6e8ceef76e33","Type":"ContainerDied","Data":"974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7"}
Jul 17 09:45:07.326070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:07.325972    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="974d50d133734217a8c17b6eeb25eb1d4a80f9d5745bf8b09c059fcb888c1eb7"
Jul 17 09:45:48.150943 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.150912    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:45:48.151404 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.151257    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9db63d64-322a-477e-b0e5-6e8ceef76e33" containerName="cleanup"
Jul 17 09:45:48.151404 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.151268    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9db63d64-322a-477e-b0e5-6e8ceef76e33" containerName="cleanup"
Jul 17 09:45:48.151404 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.151373    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9db63d64-322a-477e-b0e5-6e8ceef76e33" containerName="cleanup"
Jul 17 09:45:48.154568 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.154550    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155553 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155519    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155646    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhxxc\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.155929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.155802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.159508 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.159480    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f15c7b6a-openshift-default-dockercfg-592sm\""
Jul 17 09:45:48.173378 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.173354    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:45:48.256440 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256409    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256440 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256560    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256637    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhxxc\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256693    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.256949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.256907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.257162 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.257026    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.257269 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.257240    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.257638 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.257615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.259138 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.259115    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.259752 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.259730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.272280 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.272252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.272955 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.272931    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhxxc\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc\") pod \"e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") " pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.465714 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.465640    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:48.620384 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.620360    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:45:48.621561 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:45:48.621536    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813dafef_73a4_4f54_b4f2_934a1a129c19.slice/crio-3ebc927b56498ac7106b79445760f9da788898babe9a7de15f9dd8b89ae276e9 WatchSource:0}: Error finding container 3ebc927b56498ac7106b79445760f9da788898babe9a7de15f9dd8b89ae276e9: Status 404 returned error can't find the container with id 3ebc927b56498ac7106b79445760f9da788898babe9a7de15f9dd8b89ae276e9
Jul 17 09:45:48.623631 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.623597    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:45:48.623742 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.623672    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:45:48.623742 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:48.623719    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:45:49.528479 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:49.528443    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerStarted","Data":"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"}
Jul 17 09:45:49.528940 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:49.528498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerStarted","Data":"3ebc927b56498ac7106b79445760f9da788898babe9a7de15f9dd8b89ae276e9"}
Jul 17 09:45:49.561245 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:49.561196    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podStartSLOduration=1.561179936 podStartE2EDuration="1.561179936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:45:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:45:49.558914024 +0000 UTC m=+1662.950906640" watchObservedRunningTime="2026-07-17 09:45:49.561179936 +0000 UTC m=+1662.953172552"
Jul 17 09:45:50.465998 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:50.465965    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:50.470920 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:50.470896    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:50.533091 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:50.533054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:45:50.533788 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:45:50.533771    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:46:00.581597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.581561    2575 generic.go:356] "Generic (PLEG): container finished" podID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerID="5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac" exitCode=0
Jul 17 09:46:00.582018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.581635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerDied","Data":"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"}
Jul 17 09:46:00.582018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.581995    2575 scope.go:117] "RemoveContainer" containerID="5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"
Jul 17 09:46:00.583007 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.582977    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:46:00.583114 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.583047    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:46:00.583114 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:00.583077    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:46:01.592330 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:01.591841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerStarted","Data":"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"}
Jul 17 09:46:02.465854 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:02.465810    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:46:02.466777 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:02.466750    2575 patch_prober.go:28] interesting pod/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 17 09:46:02.466900 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:02.466799    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 17 09:46:03.466562 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:03.466521    2575 patch_prober.go:28] interesting pod/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 17 09:46:03.466972 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:03.466589    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 17 09:46:04.466925 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:04.466890    2575 patch_prober.go:28] interesting pod/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 17 09:46:04.467414 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:04.466962    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 17 09:46:05.466860 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:05.466817    2575 patch_prober.go:28] interesting pod/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 17 09:46:05.467031 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:05.466881    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 17 09:46:05.533978 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:05.533944    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:46:06.470488 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:06.470460    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:46:06.471438 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:46:06.471417    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:48:07.207128 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:07.207098    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:48:07.207902 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:07.207884    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:48:07.212394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:07.212362    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:48:07.215987 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:07.215966    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:48:54.505643 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:54.505606    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:48:54.506168 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:54.505834    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy" containerID="cri-o://45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a" gracePeriod=30
Jul 17 09:48:59.757658 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.757632    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:48:59.857126 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857098    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhxxc\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857219 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857134    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857219 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857157    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857324 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857271    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857386 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857352    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857437 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857386    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857432    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:48:59.857491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857459    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857487    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy\") pod \"813dafef-73a4-4f54-b4f2-934a1a129c19\" (UID: \"813dafef-73a4-4f54-b4f2-934a1a129c19\") "
Jul 17 09:48:59.857708 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857652    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:48:59.857770 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857755    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:48:59.857824 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857781    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:48:59.857904 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857884    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.857964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857910    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.857964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857926    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/813dafef-73a4-4f54-b4f2-934a1a129c19-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.857964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857939    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.858110 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.857992    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data" (OuterVolumeSpecName: "istio-data") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:48:59.859612 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.859589    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token" (OuterVolumeSpecName: "istio-token") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:48:59.859740 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.859649    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 09:48:59.859910 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.859881    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:48:59.860018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.859985    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc" (OuterVolumeSpecName: "kube-api-access-bhxxc") pod "813dafef-73a4-4f54-b4f2-934a1a129c19" (UID: "813dafef-73a4-4f54-b4f2-934a1a129c19"). InnerVolumeSpecName "kube-api-access-bhxxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:48:59.959394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.959362    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhxxc\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-kube-api-access-bhxxc\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.959394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.959388    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.959394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.959398    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.959616 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.959406    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:48:59.959616 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:48:59.959414    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/813dafef-73a4-4f54-b4f2-934a1a129c19-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:49:00.442057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.442022    2575 generic.go:356] "Generic (PLEG): container finished" podID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerID="45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a" exitCode=0
Jul 17 09:49:00.442229 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.442075    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"
Jul 17 09:49:00.442229 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.442108    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerDied","Data":"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"}
Jul 17 09:49:00.442229 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.442164    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm" event={"ID":"813dafef-73a4-4f54-b4f2-934a1a129c19","Type":"ContainerDied","Data":"3ebc927b56498ac7106b79445760f9da788898babe9a7de15f9dd8b89ae276e9"}
Jul 17 09:49:00.442229 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.442186    2575 scope.go:117] "RemoveContainer" containerID="45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"
Jul 17 09:49:00.451355 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.451334    2575 scope.go:117] "RemoveContainer" containerID="5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"
Jul 17 09:49:00.459859 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.459841    2575 scope.go:117] "RemoveContainer" containerID="45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"
Jul 17 09:49:00.460098 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:49:00.460078    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a\": container with ID starting with 45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a not found: ID does not exist" containerID="45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"
Jul 17 09:49:00.460156 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.460106    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a"} err="failed to get container status \"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a\": rpc error: code = NotFound desc = could not find container \"45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a\": container with ID starting with 45271800ef260fc9a75dcba218a9fb91f0d2ac2477f5b615029d56948d1b771a not found: ID does not exist"
Jul 17 09:49:00.460156 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.460124    2575 scope.go:117] "RemoveContainer" containerID="5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"
Jul 17 09:49:00.460386 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:49:00.460367    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac\": container with ID starting with 5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac not found: ID does not exist" containerID="5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"
Jul 17 09:49:00.460464 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.460391    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac"} err="failed to get container status \"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac\": rpc error: code = NotFound desc = could not find container \"5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac\": container with ID starting with 5c32c64158ac1a10ce3b40e3dae92198b8acbfe8566b70f1e4dae5334ea328ac not found: ID does not exist"
Jul 17 09:49:00.469392 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.469373    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:49:00.479162 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:00.479141    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f15c7b6a-openshift-default-7d4d5bddcc-97xjm"]
Jul 17 09:49:01.154264 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:01.154234    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" path="/var/lib/kubelet/pods/813dafef-73a4-4f54-b4f2-934a1a129c19/volumes"
Jul 17 09:49:08.347219 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.347192    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:49:08.354292 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354259    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.354444 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354293    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.354444 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354339    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.354444 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354349    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.354612 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354596    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.354649 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.354621    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="813dafef-73a4-4f54-b4f2-934a1a129c19" containerName="istio-proxy"
Jul 17 09:49:08.359564 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.359544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.364351 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.364332    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-8f9d7266-gw-openshift-default-dockercfg-vdgv6\""
Jul 17 09:49:08.433151 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433130    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433262 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433330    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433428 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433461 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433441    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433497 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj9kj\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.433571 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.433559    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.451695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.451668    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:49:08.533997 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.533961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534006    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534046    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534079    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534116    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534171 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj9kj\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534506 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534506 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534506 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534260    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534682 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534662    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534802 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534666    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.534873 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.534855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.536580 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.536551    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.536802 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.536778    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.549508 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.549478    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj9kj\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.553034 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.553011    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token\") pod \"e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") " pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.670012 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.669949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:08.821625 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.821598    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:49:08.823599 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:49:08.823575    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod835d1ac6_8afa_4737_98c1_5e0a36f113d3.slice/crio-aa6beff99fe061f581d939f914e9fdb6dc6e757ee3c936a08669a5dc1eb67f9c WatchSource:0}: Error finding container aa6beff99fe061f581d939f914e9fdb6dc6e757ee3c936a08669a5dc1eb67f9c: Status 404 returned error can't find the container with id aa6beff99fe061f581d939f914e9fdb6dc6e757ee3c936a08669a5dc1eb67f9c
Jul 17 09:49:08.825649 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.825619    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:08.825708 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.825685    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:08.825741 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:08.825714    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:09.490858 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:09.490822    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerStarted","Data":"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"}
Jul 17 09:49:09.491244 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:09.490876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerStarted","Data":"aa6beff99fe061f581d939f914e9fdb6dc6e757ee3c936a08669a5dc1eb67f9c"}
Jul 17 09:49:09.516699 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:09.516659    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podStartSLOduration=1.5166451319999998 podStartE2EDuration="1.516645132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:49:09.514444902 +0000 UTC m=+1862.906437542" watchObservedRunningTime="2026-07-17 09:49:09.516645132 +0000 UTC m=+1862.908637749"
Jul 17 09:49:09.670778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:09.670752    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:09.675341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:09.675317    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:10.496274 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:10.496238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:10.497226 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:10.497205    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:20.545098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.545061    2575 generic.go:356] "Generic (PLEG): container finished" podID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerID="4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79" exitCode=0
Jul 17 09:49:20.545527 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.545137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerDied","Data":"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"}
Jul 17 09:49:20.545603 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.545583    2575 scope.go:117] "RemoveContainer" containerID="4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"
Jul 17 09:49:20.546703 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.546670    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:20.546771 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.546753    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:20.546814 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:20.546801    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:49:21.551696 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:21.551661    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerStarted","Data":"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"}
Jul 17 09:49:21.671108 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:21.671074    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:21.671951 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:21.671926    2575 patch_prober.go:28] interesting pod/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 17 09:49:21.672051 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:21.671972    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 17 09:49:22.670273 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:22.670238    2575 patch_prober.go:28] interesting pod/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 17 09:49:22.670716 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:22.670312    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 17 09:49:23.670973 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:23.670923    2575 patch_prober.go:28] interesting pod/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 17 09:49:23.671393 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:23.670993    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 17 09:49:24.670648 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:24.670610    2575 patch_prober.go:28] interesting pod/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 17 09:49:24.670882 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:24.670670    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 17 09:49:25.497121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:25.497078    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:25.674848 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:25.674819    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:49:25.675825 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:49:25.675804    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:50:14.350774 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:14.350736    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:50:14.351212 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:14.350947    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy" containerID="cri-o://480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43" gracePeriod=30
Jul 17 09:50:19.598575 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.598550    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:50:19.674117 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674090    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674117 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674120    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674340 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674148    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674340 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj9kj\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674340 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674189    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674342    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674444    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674596 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674485    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674596 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674508    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs\") pod \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\" (UID: \"835d1ac6-8afa-4737-98c1-5e0a36f113d3\") "
Jul 17 09:50:19.674596 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674565    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:50:19.674775 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674734    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:50:19.674879 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674858    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.674943 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674884    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.674943 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674884    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:50:19.674943 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.674882    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:50:19.675345 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.675322    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:50:19.676501 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.676474    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj" (OuterVolumeSpecName: "kube-api-access-cj9kj") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "kube-api-access-cj9kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:50:19.676591 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.676523    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:50:19.676656 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.676591    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 09:50:19.677059 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.677044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "835d1ac6-8afa-4737-98c1-5e0a36f113d3" (UID: "835d1ac6-8afa-4737-98c1-5e0a36f113d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:50:19.775695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775666    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775692    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775701    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775709    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj9kj\" (UniqueName: \"kubernetes.io/projected/835d1ac6-8afa-4737-98c1-5e0a36f113d3-kube-api-access-cj9kj\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775720    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775728    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.775849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.775737    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/835d1ac6-8afa-4737-98c1-5e0a36f113d3-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:19.818679 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.818655    2575 generic.go:356] "Generic (PLEG): container finished" podID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerID="480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43" exitCode=0
Jul 17 09:50:19.818783 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.818701    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"
Jul 17 09:50:19.818783 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.818728    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerDied","Data":"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"}
Jul 17 09:50:19.818783 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.818777    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc" event={"ID":"835d1ac6-8afa-4737-98c1-5e0a36f113d3","Type":"ContainerDied","Data":"aa6beff99fe061f581d939f914e9fdb6dc6e757ee3c936a08669a5dc1eb67f9c"}
Jul 17 09:50:19.818881 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.818798    2575 scope.go:117] "RemoveContainer" containerID="480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"
Jul 17 09:50:19.827665 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.827649    2575 scope.go:117] "RemoveContainer" containerID="4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"
Jul 17 09:50:19.835070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.835054    2575 scope.go:117] "RemoveContainer" containerID="480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"
Jul 17 09:50:19.835349 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:19.835329    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43\": container with ID starting with 480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43 not found: ID does not exist" containerID="480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"
Jul 17 09:50:19.835430 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.835362    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43"} err="failed to get container status \"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43\": rpc error: code = NotFound desc = could not find container \"480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43\": container with ID starting with 480b165b7c392736370d7b568779dccf27ac5a3b4b7f207bc056944403e3ac43 not found: ID does not exist"
Jul 17 09:50:19.835430 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.835385    2575 scope.go:117] "RemoveContainer" containerID="4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"
Jul 17 09:50:19.835635 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:19.835617    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79\": container with ID starting with 4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79 not found: ID does not exist" containerID="4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"
Jul 17 09:50:19.835674 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.835642    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79"} err="failed to get container status \"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79\": rpc error: code = NotFound desc = could not find container \"4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79\": container with ID starting with 4ec74d6247feb8975d8fcb1bdfee53d2b564951ddc0e38c2f480d5acb249be79 not found: ID does not exist"
Jul 17 09:50:19.841593 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.841573    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:50:19.847796 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:19.847776    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-8f9d7266-gw-openshift-default-766f587bb7-jbcwc"]
Jul 17 09:50:21.153934 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:21.153901    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" path="/var/lib/kubelet/pods/835d1ac6-8afa-4737-98c1-5e0a36f113d3/volumes"
Jul 17 09:50:28.573430 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573398    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:50:28.573809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573746    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.573809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573758    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.573809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573778    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.573809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573783    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.573926 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573841    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.573926 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.573849    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="835d1ac6-8afa-4737-98c1-5e0a36f113d3" containerName="istio-proxy"
Jul 17 09:50:28.576838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.576818    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.579866 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.579846    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 09:50:28.582792 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.582770    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 09:50:28.582890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.582770    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-58adf1ce-serving-cert\""
Jul 17 09:50:28.582890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.582778    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 09:50:28.602692 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.602669    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:50:28.657809 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.657780    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x4dr\" (UniqueName: \"kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.657928 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.657824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.657928 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.657848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.759200 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.759165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x4dr\" (UniqueName: \"kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.759400 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.759208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.759400 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.759243    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.760024 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.759997    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.762176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.762156    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.776918 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.776892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x4dr\" (UniqueName: \"kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr\") pod \"maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:28.887062 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:28.886990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:29.041218 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:29.041186    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:50:29.044567 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:50:29.044534    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode14290b3_f5d7_4ca8_98b9_0917a03bfaa3.slice/crio-bf8a5fb338c2c940da3bf5d1f3a6245acf4643abe6ecc5f480e78575920e6a1f WatchSource:0}: Error finding container bf8a5fb338c2c940da3bf5d1f3a6245acf4643abe6ecc5f480e78575920e6a1f: Status 404 returned error can't find the container with id bf8a5fb338c2c940da3bf5d1f3a6245acf4643abe6ecc5f480e78575920e6a1f
Jul 17 09:50:29.046016 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:29.045988    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:50:29.868748 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:29.868710    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerStarted","Data":"bf8a5fb338c2c940da3bf5d1f3a6245acf4643abe6ecc5f480e78575920e6a1f"}
Jul 17 09:50:31.880574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:31.880484    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="22f569e09d607faa2cfdefb694ace8a8bfbe0238ffdcd7acb604439ec21d2174" exitCode=1
Jul 17 09:50:31.881020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:31.880573    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"22f569e09d607faa2cfdefb694ace8a8bfbe0238ffdcd7acb604439ec21d2174"}
Jul 17 09:50:31.881020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:31.880848    2575 scope.go:117] "RemoveContainer" containerID="22f569e09d607faa2cfdefb694ace8a8bfbe0238ffdcd7acb604439ec21d2174"
Jul 17 09:50:32.887252 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:32.887220    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32" exitCode=1
Jul 17 09:50:32.887896 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:32.887274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"}
Jul 17 09:50:32.887896 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:32.887345    2575 scope.go:117] "RemoveContainer" containerID="22f569e09d607faa2cfdefb694ace8a8bfbe0238ffdcd7acb604439ec21d2174"
Jul 17 09:50:32.887896 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:32.887524    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:32.887896 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:32.887726    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:50:33.349968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.349939    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:33.354442 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.354424    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:33.357219 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.357200    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:50:33.361148 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.361127    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:33.505862 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.505830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7\" (UID: \"9068aa5b-5cef-43ff-968c-b048fc4f13a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:33.606693 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.606603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7\" (UID: \"9068aa5b-5cef-43ff-968c-b048fc4f13a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:33.607230 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.607209    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7\" (UID: \"9068aa5b-5cef-43ff-968c-b048fc4f13a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:33.665739 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.665710    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:33.799066 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.799041    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:33.801280 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:50:33.801254    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9068aa5b_5cef_43ff_968c_b048fc4f13a5.slice/crio-7b98b5e7ecdb5964faed9e566cc83cbee32f6e0f47bdfacd53f02be465506a53 WatchSource:0}: Error finding container 7b98b5e7ecdb5964faed9e566cc83cbee32f6e0f47bdfacd53f02be465506a53: Status 404 returned error can't find the container with id 7b98b5e7ecdb5964faed9e566cc83cbee32f6e0f47bdfacd53f02be465506a53
Jul 17 09:50:33.887166 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.887105    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:33.893552 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.893515    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" event={"ID":"9068aa5b-5cef-43ff-968c-b048fc4f13a5","Type":"ContainerStarted","Data":"7b98b5e7ecdb5964faed9e566cc83cbee32f6e0f47bdfacd53f02be465506a53"}
Jul 17 09:50:33.895146 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:33.895131    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:33.895382 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:33.895289    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:50:34.665208 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.665125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:34.668507 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.668491    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:34.677640 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.677616    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:34.818635 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.818599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl\" (UID: \"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:34.900651 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.900623    2575 generic.go:356] "Generic (PLEG): container finished" podID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerID="b7c1dd572c1798a905b57383385b933736b13c24c36057231d5d33558a9b6890" exitCode=7
Jul 17 09:50:34.901027 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.900750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" event={"ID":"9068aa5b-5cef-43ff-968c-b048fc4f13a5","Type":"ContainerDied","Data":"b7c1dd572c1798a905b57383385b933736b13c24c36057231d5d33558a9b6890"}
Jul 17 09:50:34.901027 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.900926    2575 scope.go:117] "RemoveContainer" containerID="b7c1dd572c1798a905b57383385b933736b13c24c36057231d5d33558a9b6890"
Jul 17 09:50:34.901229 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.901212    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:34.901653 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:34.901427    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:50:34.919408 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.919351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl\" (UID: \"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:34.920072 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.920037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl\" (UID: \"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:34.979394 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:34.979367    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:35.109847 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.109822    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:35.111491 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:50:35.111465    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22c2948_dcfa_41a1_96fb_3d3430b5fbc0.slice/crio-82da88576f7f4e637e7a6f2eb7f7a75c50ba4557f7529724eba082539f2df75f WatchSource:0}: Error finding container 82da88576f7f4e637e7a6f2eb7f7a75c50ba4557f7529724eba082539f2df75f: Status 404 returned error can't find the container with id 82da88576f7f4e637e7a6f2eb7f7a75c50ba4557f7529724eba082539f2df75f
Jul 17 09:50:35.907242 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.907149    2575 generic.go:356] "Generic (PLEG): container finished" podID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerID="6a8b45c98ea4fc26f8f030a8a539f5a925f63f569a98293d69efded647e946e0" exitCode=6
Jul 17 09:50:35.907694 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.907238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" event={"ID":"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0","Type":"ContainerDied","Data":"6a8b45c98ea4fc26f8f030a8a539f5a925f63f569a98293d69efded647e946e0"}
Jul 17 09:50:35.907694 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.907315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" event={"ID":"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0","Type":"ContainerStarted","Data":"82da88576f7f4e637e7a6f2eb7f7a75c50ba4557f7529724eba082539f2df75f"}
Jul 17 09:50:35.907694 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.907499    2575 scope.go:117] "RemoveContainer" containerID="6a8b45c98ea4fc26f8f030a8a539f5a925f63f569a98293d69efded647e946e0"
Jul 17 09:50:35.909382 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.909356    2575 generic.go:356] "Generic (PLEG): container finished" podID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerID="10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8" exitCode=7
Jul 17 09:50:35.909485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.909389    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" event={"ID":"9068aa5b-5cef-43ff-968c-b048fc4f13a5","Type":"ContainerDied","Data":"10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8"}
Jul 17 09:50:35.909485 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.909432    2575 scope.go:117] "RemoveContainer" containerID="b7c1dd572c1798a905b57383385b933736b13c24c36057231d5d33558a9b6890"
Jul 17 09:50:35.909609 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:35.909594    2575 scope.go:117] "RemoveContainer" containerID="10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8"
Jul 17 09:50:35.909832 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:35.909803    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7_odh-ai-gateway-infra(9068aa5b-5cef-43ff-968c-b048fc4f13a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5"
Jul 17 09:50:36.916091 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:36.916005    2575 scope.go:117] "RemoveContainer" containerID="10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8"
Jul 17 09:50:36.916497 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:36.916231    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7_odh-ai-gateway-infra(9068aa5b-5cef-43ff-968c-b048fc4f13a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5"
Jul 17 09:50:36.917569 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:36.917543    2575 generic.go:356] "Generic (PLEG): container finished" podID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerID="a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4" exitCode=6
Jul 17 09:50:36.917677 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:36.917595    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" event={"ID":"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0","Type":"ContainerDied","Data":"a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4"}
Jul 17 09:50:36.917677 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:36.917643    2575 scope.go:117] "RemoveContainer" containerID="6a8b45c98ea4fc26f8f030a8a539f5a925f63f569a98293d69efded647e946e0"
Jul 17 09:50:36.917787 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:36.917771    2575 scope.go:117] "RemoveContainer" containerID="a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4"
Jul 17 09:50:36.917968 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:36.917950    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl_odh-ai-gateway-infra(c22c2948-dcfa-41a1-96fb-3d3430b5fbc0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0"
Jul 17 09:50:37.924220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:37.924189    2575 scope.go:117] "RemoveContainer" containerID="a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4"
Jul 17 09:50:37.924655 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:37.924418    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl_odh-ai-gateway-infra(c22c2948-dcfa-41a1-96fb-3d3430b5fbc0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0"
Jul 17 09:50:38.887989 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:38.887951    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:38.888386 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:38.888371    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:38.888569 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:38.888546    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:50:49.147573 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:49.147541    2575 scope.go:117] "RemoveContainer" containerID="10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8"
Jul 17 09:50:49.983607 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:49.983575    2575 generic.go:356] "Generic (PLEG): container finished" podID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerID="3db547fa41f8c42f423a8c92c024258a36da5cd09123dc5db5c93d8a22b30d67" exitCode=7
Jul 17 09:50:49.983762 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:49.983651    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" event={"ID":"9068aa5b-5cef-43ff-968c-b048fc4f13a5","Type":"ContainerDied","Data":"3db547fa41f8c42f423a8c92c024258a36da5cd09123dc5db5c93d8a22b30d67"}
Jul 17 09:50:49.983762 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:49.983714    2575 scope.go:117] "RemoveContainer" containerID="10111423b4b87fa935712e90917d48f568178ba8b6814c0611b3ff75f4b9e2b8"
Jul 17 09:50:49.984056 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:49.984032    2575 scope.go:117] "RemoveContainer" containerID="3db547fa41f8c42f423a8c92c024258a36da5cd09123dc5db5c93d8a22b30d67"
Jul 17 09:50:49.984263 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:49.984240    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7_odh-ai-gateway-infra(9068aa5b-5cef-43ff-968c-b048fc4f13a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5"
Jul 17 09:50:51.012686 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.012653    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:51.144338 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.144312    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:51.255737 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.255706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca\") pod \"9068aa5b-5cef-43ff-968c-b048fc4f13a5\" (UID: \"9068aa5b-5cef-43ff-968c-b048fc4f13a5\") "
Jul 17 09:50:51.256057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.256032    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9068aa5b-5cef-43ff-968c-b048fc4f13a5" (UID: "9068aa5b-5cef-43ff-968c-b048fc4f13a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:50:51.356701 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.356613    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9068aa5b-5cef-43ff-968c-b048fc4f13a5-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:51.996121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.996092    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"
Jul 17 09:50:51.996121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.996107    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7" event={"ID":"9068aa5b-5cef-43ff-968c-b048fc4f13a5","Type":"ContainerDied","Data":"7b98b5e7ecdb5964faed9e566cc83cbee32f6e0f47bdfacd53f02be465506a53"}
Jul 17 09:50:51.996382 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:51.996161    2575 scope.go:117] "RemoveContainer" containerID="3db547fa41f8c42f423a8c92c024258a36da5cd09123dc5db5c93d8a22b30d67"
Jul 17 09:50:52.018362 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:52.018339    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:52.024377 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:52.024356    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-nkdz7"]
Jul 17 09:50:53.147489 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:53.147457    2575 scope.go:117] "RemoveContainer" containerID="a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4"
Jul 17 09:50:53.153687 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:53.153624    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" path="/var/lib/kubelet/pods/9068aa5b-5cef-43ff-968c-b048fc4f13a5/volumes"
Jul 17 09:50:54.007503 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:54.007469    2575 generic.go:356] "Generic (PLEG): container finished" podID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerID="256fc31d1b6b492f08ed85bb6cb9662131c34634b02981c692d765a866f12f64" exitCode=6
Jul 17 09:50:54.007642 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:54.007529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" event={"ID":"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0","Type":"ContainerDied","Data":"256fc31d1b6b492f08ed85bb6cb9662131c34634b02981c692d765a866f12f64"}
Jul 17 09:50:54.007642 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:54.007579    2575 scope.go:117] "RemoveContainer" containerID="a1af93afc998f356047eee43179ab67a3bf004f575592fb74380b49cef3c0ab4"
Jul 17 09:50:54.007866 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:54.007845    2575 scope.go:117] "RemoveContainer" containerID="256fc31d1b6b492f08ed85bb6cb9662131c34634b02981c692d765a866f12f64"
Jul 17 09:50:54.008103 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:54.008083    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl_odh-ai-gateway-infra(c22c2948-dcfa-41a1-96fb-3d3430b5fbc0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0"
Jul 17 09:50:54.148070 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:54.147995    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:55.013698 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.013667    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4" exitCode=1
Jul 17 09:50:55.013867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.013752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"}
Jul 17 09:50:55.013867 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.013816    2575 scope.go:117] "RemoveContainer" containerID="180fb2fb73de5d8701cdc9841f245cb4840a0398393e1ca7f6a28ff48643ef32"
Jul 17 09:50:55.014212 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.014187    2575 scope.go:117] "RemoveContainer" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"
Jul 17 09:50:55.014457 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:55.014436    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:50:55.038241 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.038216    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:55.183140 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.183117    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:55.290578 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.290546    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca\") pod \"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0\" (UID: \"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0\") "
Jul 17 09:50:55.290947 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.290925    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" (UID: "c22c2948-dcfa-41a1-96fb-3d3430b5fbc0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:50:55.391395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:55.391368    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:50:56.023727 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:56.023694    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl" event={"ID":"c22c2948-dcfa-41a1-96fb-3d3430b5fbc0","Type":"ContainerDied","Data":"82da88576f7f4e637e7a6f2eb7f7a75c50ba4557f7529724eba082539f2df75f"}
Jul 17 09:50:56.023919 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:56.023749    2575 scope.go:117] "RemoveContainer" containerID="256fc31d1b6b492f08ed85bb6cb9662131c34634b02981c692d765a866f12f64"
Jul 17 09:50:56.023919 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:56.023701    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"
Jul 17 09:50:56.046320 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:56.046282    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:56.054187 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:56.054165    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-rgtwl"]
Jul 17 09:50:57.153493 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:57.153450    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" path="/var/lib/kubelet/pods/c22c2948-dcfa-41a1-96fb-3d3430b5fbc0/volumes"
Jul 17 09:50:58.887282 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:58.887249    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:58.887798 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:58.887331    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:50:58.887798 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:50:58.887730    2575 scope.go:117] "RemoveContainer" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"
Jul 17 09:50:58.887980 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:50:58.887956    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:10.147869 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:10.147839    2575 scope.go:117] "RemoveContainer" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"
Jul 17 09:51:10.148251 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:10.148027    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:22.147497 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:22.147469    2575 scope.go:117] "RemoveContainer" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"
Jul 17 09:51:23.151277 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:23.151193    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302" exitCode=1
Jul 17 09:51:23.152476 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:23.152453    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"}
Jul 17 09:51:23.152543 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:23.152510    2575 scope.go:117] "RemoveContainer" containerID="d595558401a2d9a14de0900f00c8137c5a65eb1cf1f88441c5b5007374a2b8e4"
Jul 17 09:51:23.152825 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:23.152808    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:51:23.153021 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:23.153001    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:23.887426 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:23.887389    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:51:24.157033 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:24.156961    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:51:24.157452 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:24.157125    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:28.887998 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:28.887966    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:51:28.888408 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:28.888293    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:51:28.888504 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:28.888484    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:41.147565 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:41.147532    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:51:41.148042 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:41.147778    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:51:52.147552 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:51:52.147519    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:51:52.147952 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:51:52.147702    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:52:05.148225 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:05.148192    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:52:06.360491 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:06.360462    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d" exitCode=1
Jul 17 09:52:06.360924 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:06.360539    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"}
Jul 17 09:52:06.360924 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:06.360616    2575 scope.go:117] "RemoveContainer" containerID="57a3d3fa6d146829e4e5e14d0ca14b6fd1e4fef26e8c62b95e05c2f410af0302"
Jul 17 09:52:06.361037 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:06.360963    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:52:06.361182 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:52:06.361159    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:52:08.887743 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:08.887708    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:52:08.888182 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:08.887764    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:52:08.888182 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:08.888046    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:52:08.888279 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:52:08.888210    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:52:24.147984 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:24.147951    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:52:24.148638 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:52:24.148218    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:52:37.156175 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:37.156101    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:52:37.156738 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:52:37.156317    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:52:50.148050 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:52:50.148019    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:52:50.148555 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:52:50.148199    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:05.147281 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:05.147250    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:53:05.147777 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:05.147463    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:07.240728 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:07.240698    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:53:07.242452 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:07.242434    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:53:07.246026 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:07.246003    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:53:07.247456 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:07.247439    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:53:20.147905 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:20.147867    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:53:20.148293 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:20.148097    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:35.147988 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:35.147952    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:53:35.777443 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:35.777411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerStarted","Data":"d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"}
Jul 17 09:53:35.777650 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:35.777622    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:53:35.802683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:35.802633    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podStartSLOduration=183.579662965 podStartE2EDuration="3m7.802618389s" totalImagesPullingTime="4.222955424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:50:28 +0000 UTC" imagePullSessionsCount=6 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:53:35.799410529 +0000 UTC m=+2129.191403157" watchObservedRunningTime="2026-07-17 09:53:35.802618389 +0000 UTC m=+2129.194611042"
Jul 17 09:53:36.786565 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:36.786531    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82" exitCode=1
Jul 17 09:53:36.787077 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:36.786598    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"}
Jul 17 09:53:36.787077 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:36.786848    2575 scope.go:117] "RemoveContainer" containerID="d8edd214d77c74823dc4d7de3690d3edadf1f6b677a5eb858bbc57ea38a1eb8d"
Jul 17 09:53:36.787470 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:36.787447    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:53:36.787844 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:36.787813    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:37.792057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:37.792030    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:53:37.792458 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:37.792221    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:38.887398 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:38.887368    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:53:38.887779 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:38.887697    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:53:38.887887 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:38.887866    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:53:52.148093 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:53:52.148062    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:53:52.148487 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:53:52.148237    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:54:04.147727 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:54:04.147657    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:54:04.148195 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:54:04.147827    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:54:16.147565 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:54:16.147531    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:54:16.148015 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:54:16.147698    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:54:31.148030 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:54:31.147997    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:54:31.150519 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:54:31.148229    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:54:43.148018 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:54:43.147986    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:54:43.148654 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:54:43.148141    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:54:55.148109 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:54:55.148077    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:54:55.148602 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:54:55.148253    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:55:07.150289 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:55:07.150252    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:55:07.150822 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:55:07.150480    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:55:20.147993 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:55:20.147961    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:55:20.148426 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:55:20.148143    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:55:34.147852 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:55:34.147769    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:55:34.148226 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:55:34.147954    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:55:48.147119 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:55:48.147086    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:55:48.147607 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:55:48.147268    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:55:59.147964 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:55:59.147929    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:55:59.148475 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:55:59.148165    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:56:13.147277 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:13.147239    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:56:13.147775 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:13.147517    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:56:27.150617 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:27.150587    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:56:27.151022 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:27.150860    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:56:28.593777 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:28.593748    2575 generic.go:356] "Generic (PLEG): container finished" podID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerID="af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27" exitCode=1
Jul 17 09:56:28.594220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:28.593823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27"}
Jul 17 09:56:28.594220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:28.593895    2575 scope.go:117] "RemoveContainer" containerID="d5e86a1bd43380da7434214994512eecaa689fe7f7c5d99deba2729fbe621e82"
Jul 17 09:56:28.594220 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:28.594204    2575 scope.go:117] "RemoveContainer" containerID="af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27"
Jul 17 09:56:28.594426 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:28.594406    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:56:28.887387 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:28.887282    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:56:29.600853 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:29.600830    2575 scope.go:117] "RemoveContainer" containerID="af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27"
Jul 17 09:56:29.601283 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:29.600994    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:56:30.777921 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:30.777894    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:56:30.778389 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:30.778226    2575 scope.go:117] "RemoveContainer" containerID="af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27"
Jul 17 09:56:30.778446 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:30.778426    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=maas-api pod=maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q_odh-ai-gateway-infra(e14290b3-f5d7-4ca8-98b9-0917a03bfaa3)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"
Jul 17 09:56:39.791421 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:39.791385    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:56:39.919645 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:39.919623    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:56:40.022193 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.022161    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls\") pod \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") "
Jul 17 09:56:40.022381 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.022209    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6x4dr\" (UniqueName: \"kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr\") pod \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") "
Jul 17 09:56:40.022381 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.022272    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca\") pod \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\" (UID: \"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3\") "
Jul 17 09:56:40.022710 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.022689    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" (UID: "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:40.024508 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.024486    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr" (OuterVolumeSpecName: "kube-api-access-6x4dr") pod "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" (UID: "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"). InnerVolumeSpecName "kube-api-access-6x4dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:56:40.024599 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.024505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" (UID: "e14290b3-f5d7-4ca8-98b9-0917a03bfaa3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:56:40.123143 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.123075    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:56:40.123143 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.123100    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:56:40.123143 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.123109    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x4dr\" (UniqueName: \"kubernetes.io/projected/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3-kube-api-access-6x4dr\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:56:40.654008 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.653974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q" event={"ID":"e14290b3-f5d7-4ca8-98b9-0917a03bfaa3","Type":"ContainerDied","Data":"bf8a5fb338c2c940da3bf5d1f3a6245acf4643abe6ecc5f480e78575920e6a1f"}
Jul 17 09:56:40.654201 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.654035    2575 scope.go:117] "RemoveContainer" containerID="af3adb936263bfd868978248d0dd2082cce2de12015695cdba4c02e2408bff27"
Jul 17 09:56:40.654201 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.653987    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"
Jul 17 09:56:40.676597 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.676572    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:56:40.681768 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:40.681745    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-58adf1ce-66677b96fc-lxx2q"]
Jul 17 09:56:41.152734 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:41.152701    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" path="/var/lib/kubelet/pods/e14290b3-f5d7-4ca8-98b9-0917a03bfaa3/volumes"
Jul 17 09:56:44.744399 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744362    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744693    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744704    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744714    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744719    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744725    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744731    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744738    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744744    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744752    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744757    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744764    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744770    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744776    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.744778 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744781    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744788    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744793    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744800    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744805    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744815    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744820    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744828    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744833    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744901    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744911    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744918    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744923    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744929    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744934    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744938    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744943    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:44.745204 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.744948    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:44.747952 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.747936    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:56:44.750624 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.750604    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 09:56:44.750756 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.750678    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:56:44.758611 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.758589    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:56:44.860511 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.860482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74\" (UID: \"d8fa0878-bfef-4dcb-b0be-cab2cc156402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:56:44.961722 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.961691    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74\" (UID: \"d8fa0878-bfef-4dcb-b0be-cab2cc156402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:56:44.962321 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:44.962274    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74\" (UID: \"d8fa0878-bfef-4dcb-b0be-cab2cc156402\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:56:45.058910 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:45.058838    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:56:45.191652 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:45.191626    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:56:45.193244 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:56:45.193217    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8fa0878_bfef_4dcb_b0be_cab2cc156402.slice/crio-734d15784a35fe19b4d3f7766184caeaa69dfea75915d4f8c2955fdb10ec6320 WatchSource:0}: Error finding container 734d15784a35fe19b4d3f7766184caeaa69dfea75915d4f8c2955fdb10ec6320: Status 404 returned error can't find the container with id 734d15784a35fe19b4d3f7766184caeaa69dfea75915d4f8c2955fdb10ec6320
Jul 17 09:56:45.678665 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:45.678573    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerStarted","Data":"23dadf308f82a3c0af88db64bbafa4ef45fb204bf2a814fa0a74cac46abfc05a"}
Jul 17 09:56:45.678665 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:45.678642    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerStarted","Data":"734d15784a35fe19b4d3f7766184caeaa69dfea75915d4f8c2955fdb10ec6320"}
Jul 17 09:56:45.716460 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:45.716417    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" podStartSLOduration=1.716402214 podStartE2EDuration="1.716402214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:56:45.713965664 +0000 UTC m=+2319.105958281" watchObservedRunningTime="2026-07-17 09:56:45.716402214 +0000 UTC m=+2319.108394831"
Jul 17 09:56:46.684333 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:46.684292    2575 generic.go:356] "Generic (PLEG): container finished" podID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerID="23dadf308f82a3c0af88db64bbafa4ef45fb204bf2a814fa0a74cac46abfc05a" exitCode=6
Jul 17 09:56:46.684753 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:46.684354    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerDied","Data":"23dadf308f82a3c0af88db64bbafa4ef45fb204bf2a814fa0a74cac46abfc05a"}
Jul 17 09:56:46.684753 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:46.684537    2575 scope.go:117] "RemoveContainer" containerID="23dadf308f82a3c0af88db64bbafa4ef45fb204bf2a814fa0a74cac46abfc05a"
Jul 17 09:56:47.018453 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018427    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:56:47.018796 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018783    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:47.018838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018798    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:47.018838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018809    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:47.018838 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018814    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:47.018932 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018867    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9068aa5b-5cef-43ff-968c-b048fc4f13a5" containerName="revoke-keys"
Jul 17 09:56:47.018932 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018877    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22c2948-dcfa-41a1-96fb-3d3430b5fbc0" containerName="revoke-keys"
Jul 17 09:56:47.018932 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018884    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:47.018932 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.018890    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e14290b3-f5d7-4ca8-98b9-0917a03bfaa3" containerName="maas-api"
Jul 17 09:56:47.021763 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.021748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.025107 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.025080    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5593c7b6-openshift-default-dockercfg-vltzq\""
Jul 17 09:56:47.081046 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081018    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:56:47.081397 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081369    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd8dl\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081507 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081588 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081588 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081585    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081636    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081707 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.081866 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.081716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182530 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182496    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd8dl\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182559    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182591    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182609    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182695 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182993 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182702    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.182993 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.182730    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.183103 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.183032    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.183176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.183158    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.183236 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.183175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.183236 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.183211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.191952 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.184237    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.191952 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.185399    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.192771 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.192738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd8dl\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.192873 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.192802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.193039 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.193018    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token\") pod \"e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") " pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.332108 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.332083    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:47.477034 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.477001    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:56:47.483133 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:56:47.483106    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ca3ed0_9149_4d11_87a8_6dd6200249cf.slice/crio-e1c0b35797c4af9b311529f745a98dd19fba3e4b837e90a3eee4d27bd67c2017 WatchSource:0}: Error finding container e1c0b35797c4af9b311529f745a98dd19fba3e4b837e90a3eee4d27bd67c2017: Status 404 returned error can't find the container with id e1c0b35797c4af9b311529f745a98dd19fba3e4b837e90a3eee4d27bd67c2017
Jul 17 09:56:47.485254 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.485220    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:47.485341 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.485290    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:47.485388 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.485353    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:47.692833 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.692796    2575 generic.go:356] "Generic (PLEG): container finished" podID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerID="29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2" exitCode=6
Jul 17 09:56:47.693329 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.692835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerDied","Data":"29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2"}
Jul 17 09:56:47.693329 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.692912    2575 scope.go:117] "RemoveContainer" containerID="23dadf308f82a3c0af88db64bbafa4ef45fb204bf2a814fa0a74cac46abfc05a"
Jul 17 09:56:47.693329 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.693216    2575 scope.go:117] "RemoveContainer" containerID="29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2"
Jul 17 09:56:47.693518 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:47.693468    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74_odh-ai-gateway-infra(d8fa0878-bfef-4dcb-b0be-cab2cc156402)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402"
Jul 17 09:56:47.694934 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.694911    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerStarted","Data":"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"}
Jul 17 09:56:47.695043 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.694959    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerStarted","Data":"e1c0b35797c4af9b311529f745a98dd19fba3e4b837e90a3eee4d27bd67c2017"}
Jul 17 09:56:47.731256 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:47.731215    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podStartSLOduration=1.731199291 podStartE2EDuration="1.731199291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:56:47.730528624 +0000 UTC m=+2321.122521254" watchObservedRunningTime="2026-07-17 09:56:47.731199291 +0000 UTC m=+2321.123191912"
Jul 17 09:56:48.332798 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.332768    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:48.338497 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.338468    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:48.345125 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.345096    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 09:56:48.348929 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.348908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.351656 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.351636    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-5593c7b6-serving-cert\""
Jul 17 09:56:48.351786 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.351708    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 09:56:48.351939 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.351925    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 09:56:48.364806 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.364785    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 09:56:48.495009 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.494966    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.495175 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.495085    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmn8\" (UniqueName: \"kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.495175 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.495131    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.595827 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.595762    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmn8\" (UniqueName: \"kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.595827 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.595804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.595984 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.595838    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.595984 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:48.595922    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-serving-cert: secret "maas-api-e2e-derive-5593c7b6-serving-cert" not found
Jul 17 09:56:48.596051 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:48.595990    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls podName:87957597-5eac-4186-a47c-eaca95614a40 nodeName:}" failed. No retries permitted until 2026-07-17 09:56:49.095971979 +0000 UTC m=+2322.487964575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls") pod "maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" (UID: "87957597-5eac-4186-a47c-eaca95614a40") : secret "maas-api-e2e-derive-5593c7b6-serving-cert" not found
Jul 17 09:56:48.596547 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.596528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.606093 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.606076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmn8\" (UniqueName: \"kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:48.701196 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.701171    2575 scope.go:117] "RemoveContainer" containerID="29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2"
Jul 17 09:56:48.701619 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.701315    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:48.701619 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:48.701410    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74_odh-ai-gateway-infra(d8fa0878-bfef-4dcb-b0be-cab2cc156402)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402"
Jul 17 09:56:48.702254 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:48.702232    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:56:49.100258 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:49.100230    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:49.102683 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:49.102662    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") pod \"maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:49.260652 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:49.260617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:49.604770 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:49.604737    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 09:56:49.607618 ip-10-0-139-140 kubenswrapper[2575]: W0717 09:56:49.607582    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87957597_5eac_4186_a47c_eaca95614a40.slice/crio-a36361762a3a3f406e550ab6801be80137d00a2888e776713a9e3d1c009cf4d6 WatchSource:0}: Error finding container a36361762a3a3f406e550ab6801be80137d00a2888e776713a9e3d1c009cf4d6: Status 404 returned error can't find the container with id a36361762a3a3f406e550ab6801be80137d00a2888e776713a9e3d1c009cf4d6
Jul 17 09:56:49.707885 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:49.707855    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerStarted","Data":"a36361762a3a3f406e550ab6801be80137d00a2888e776713a9e3d1c009cf4d6"}
Jul 17 09:56:50.713762 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:50.713684    2575 generic.go:356] "Generic (PLEG): container finished" podID="87957597-5eac-4186-a47c-eaca95614a40" containerID="4da6790d9f4714e5b5d1620ec416eed0b0f62e158ed03c972d10de27f8aa322d" exitCode=1
Jul 17 09:56:50.714098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:50.713767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"4da6790d9f4714e5b5d1620ec416eed0b0f62e158ed03c972d10de27f8aa322d"}
Jul 17 09:56:50.714098 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:50.713927    2575 scope.go:117] "RemoveContainer" containerID="4da6790d9f4714e5b5d1620ec416eed0b0f62e158ed03c972d10de27f8aa322d"
Jul 17 09:56:51.722582 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:51.722547    2575 generic.go:356] "Generic (PLEG): container finished" podID="87957597-5eac-4186-a47c-eaca95614a40" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d" exitCode=1
Jul 17 09:56:51.723057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:51.722632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"}
Jul 17 09:56:51.723057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:51.722713    2575 scope.go:117] "RemoveContainer" containerID="4da6790d9f4714e5b5d1620ec416eed0b0f62e158ed03c972d10de27f8aa322d"
Jul 17 09:56:51.723057 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:51.722840    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:56:51.723187 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:51.723068    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:56:52.729165 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:52.729135    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:56:52.729585 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:52.729326    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:56:54.260760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:54.260726    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:54.261149 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:54.261100    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:56:54.261290 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:54.261269    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:56:58.758021 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.757946    2575 generic.go:356] "Generic (PLEG): container finished" podID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerID="44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94" exitCode=0
Jul 17 09:56:58.758434 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.758018    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerDied","Data":"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"}
Jul 17 09:56:58.758481 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.758428    2575 scope.go:117] "RemoveContainer" containerID="44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"
Jul 17 09:56:58.759477 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.759445    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:58.759599 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.759512    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:58.759599 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:58.759543    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 09:56:59.260864 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:59.260833    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:56:59.261205 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:59.261192    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:56:59.261417 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:56:59.261396    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:56:59.764344 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:56:59.764289    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerStarted","Data":"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"}
Jul 17 09:57:00.332681 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:00.332642    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:57:00.333928 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:00.333898    2575 patch_prober.go:28] interesting pod/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 17 09:57:00.334064 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:00.333961    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 17 09:57:01.333409 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:01.333366    2575 patch_prober.go:28] interesting pod/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 17 09:57:01.333887 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:01.333429    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 17 09:57:02.147771 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:02.147739    2575 scope.go:117] "RemoveContainer" containerID="29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2"
Jul 17 09:57:02.333310 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:02.333272    2575 patch_prober.go:28] interesting pod/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 17 09:57:02.333493 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:02.333363    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 17 09:57:02.782494 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:02.782395    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerStarted","Data":"b166d238161b96250a9063a15ba6f4728ee020794671bbc9bb8c31034a1ec3e7"}
Jul 17 09:57:03.332904 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.332871    2575 patch_prober.go:28] interesting pod/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 17 09:57:03.333087 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.332925    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 17 09:57:03.701814 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.701722    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:57:03.788933 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.788899    2575 generic.go:356] "Generic (PLEG): container finished" podID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerID="b166d238161b96250a9063a15ba6f4728ee020794671bbc9bb8c31034a1ec3e7" exitCode=6
Jul 17 09:57:03.788933 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.788936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerDied","Data":"b166d238161b96250a9063a15ba6f4728ee020794671bbc9bb8c31034a1ec3e7"}
Jul 17 09:57:03.789192 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.788989    2575 scope.go:117] "RemoveContainer" containerID="29def33535a2617f2a6d94fabfea1ebccc9bcf7a9076717165dfbb13554001c2"
Jul 17 09:57:03.789337 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.789291    2575 scope.go:117] "RemoveContainer" containerID="b166d238161b96250a9063a15ba6f4728ee020794671bbc9bb8c31034a1ec3e7"
Jul 17 09:57:03.789624 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:03.789599    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74_odh-ai-gateway-infra(d8fa0878-bfef-4dcb-b0be-cab2cc156402)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402"
Jul 17 09:57:03.812290 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:03.812265    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:57:04.337484 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:04.337443    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:57:04.338316 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:04.338275    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:57:04.931078 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:04.931054    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:57:05.048802 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.048717    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca\") pod \"d8fa0878-bfef-4dcb-b0be-cab2cc156402\" (UID: \"d8fa0878-bfef-4dcb-b0be-cab2cc156402\") "
Jul 17 09:57:05.049063 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.049040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8fa0878-bfef-4dcb-b0be-cab2cc156402" (UID: "d8fa0878-bfef-4dcb-b0be-cab2cc156402"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:05.149384 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.149349    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8fa0878-bfef-4dcb-b0be-cab2cc156402-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:57:05.803917 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.803888    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"
Jul 17 09:57:05.804074 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.803927    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74" event={"ID":"d8fa0878-bfef-4dcb-b0be-cab2cc156402","Type":"ContainerDied","Data":"734d15784a35fe19b4d3f7766184caeaa69dfea75915d4f8c2955fdb10ec6320"}
Jul 17 09:57:05.804074 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.803984    2575 scope.go:117] "RemoveContainer" containerID="b166d238161b96250a9063a15ba6f4728ee020794671bbc9bb8c31034a1ec3e7"
Jul 17 09:57:05.822763 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.822735    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:57:05.830430 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:05.830409    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wnn74"]
Jul 17 09:57:07.154487 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:07.154456    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" path="/var/lib/kubelet/pods/d8fa0878-bfef-4dcb-b0be-cab2cc156402/volumes"
Jul 17 09:57:13.148117 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:13.148077    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:57:13.843978 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:13.843905    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerStarted","Data":"ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"}
Jul 17 09:57:13.844120 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:13.844107    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:57:13.863939 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:13.863891    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podStartSLOduration=24.694115138 podStartE2EDuration="25.863874922s" totalImagesPullingTime="1.169759784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:56:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:57:13.862640386 +0000 UTC m=+2347.254633004" watchObservedRunningTime="2026-07-17 09:57:13.863874922 +0000 UTC m=+2347.255867592"
Jul 17 09:57:14.849976 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:14.849947    2575 generic.go:356] "Generic (PLEG): container finished" podID="87957597-5eac-4186-a47c-eaca95614a40" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c" exitCode=1
Jul 17 09:57:14.850423 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:14.849991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"}
Jul 17 09:57:14.850423 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:14.850081    2575 scope.go:117] "RemoveContainer" containerID="aebb0639c7bd7d84fb4cdb8b850b0974f8408ee9f3765d3c57ac047fcbe0514d"
Jul 17 09:57:14.850423 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:14.850322    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:14.850554 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:14.850537    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:15.855926 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:15.855898    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:15.856326 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:15.856076    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:19.261765 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:19.261732    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:57:19.262156 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:19.262088    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:19.262276 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:19.262254    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:33.147828 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:33.147795    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:33.148314 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:33.148033    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:45.147949 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:45.147916    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:45.998362 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:45.998329    2575 generic.go:356] "Generic (PLEG): container finished" podID="87957597-5eac-4186-a47c-eaca95614a40" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04" exitCode=1
Jul 17 09:57:45.998523 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:45.998396    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"}
Jul 17 09:57:45.998523 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:45.998455    2575 scope.go:117] "RemoveContainer" containerID="ddb0181414916dba28c1c2207c3f9c590bf901949dd1855ff13332cc6d568e2c"
Jul 17 09:57:45.998818 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:45.998803    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:57:45.999049 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:45.999027    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:48.844636 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:48.844597    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:57:48.845020 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:48.844984    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:57:48.845178 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:48.845157    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:57:49.260766 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:49.260738    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:57:49.261181 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:57:49.261166    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:57:49.261429 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:57:49.261405    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:00.148135 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:00.148101    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:58:00.148548 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:00.148263    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:07.269644 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:07.269610    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:58:07.272395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:07.272374    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 09:58:07.275053 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:07.275029    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:58:07.277849 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:07.277827    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 09:58:14.147519 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:14.147493    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:58:14.147922 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:14.147689    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:25.148035 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:25.148009    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:58:25.148457 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:25.148197    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:40.147572 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:40.147489    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:58:41.266092 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:41.266056    2575 generic.go:356] "Generic (PLEG): container finished" podID="87957597-5eac-4186-a47c-eaca95614a40" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889" exitCode=1
Jul 17 09:58:41.266092 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:41.266095    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"}
Jul 17 09:58:41.266563 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:41.266148    2575 scope.go:117] "RemoveContainer" containerID="784705b605b31aec131dd8ec996bc35bcec00e5572f5e2a620c7a0dc44a8ea04"
Jul 17 09:58:41.266563 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:41.266457    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:58:41.266712 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:41.266687    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:43.844577 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:43.844538    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:58:43.845000 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:43.844856    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:58:43.845056 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:43.845019    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:58:49.261526 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:49.261496    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:58:49.261898 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:58:49.261885    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:58:49.262092 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:58:49.262072    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:03.147986 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:03.147952    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:03.148417 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:03.148145    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:14.147719 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:14.147686    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:14.148451 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:14.147906    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:28.147900 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:28.147866    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:28.148397 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:28.148106    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:39.153529 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:39.153503    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:39.153934 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:39.153676    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:51.148116 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:51.148084    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:51.148626 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:51.148280    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq_odh-ai-gateway-infra(87957597-5eac-4186-a47c-eaca95614a40)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" podUID="87957597-5eac-4186-a47c-eaca95614a40"
Jul 17 09:59:52.761059 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:52.761025    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:59:52.761493 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:52.761224    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy" containerID="cri-o://4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6" gracePeriod=30
Jul 17 09:59:58.022073 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.022011    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:59:58.023082 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023064    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023126 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023108    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023172 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023126    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023172 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023150    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023243 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023186    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023243 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023221    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023345 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023249    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd8dl\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023345 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023275    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023459 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023363    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket\") pod \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\" (UID: \"70ca3ed0-9149-4d11-87a8-6dd6200249cf\") "
Jul 17 09:59:58.023733 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023537    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:59:58.023733 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023555    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:58.023890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023755    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:59:58.023890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023778    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.023890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023799    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.023890 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.023815    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:59:58.024156 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.024130    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:59:58.025395 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.025367    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:59:58.025504 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.025431    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:59:58.025574 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.025555    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 09:59:58.025797 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.025776    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl" (OuterVolumeSpecName: "kube-api-access-dd8dl") pod "70ca3ed0-9149-4d11-87a8-6dd6200249cf" (UID: "70ca3ed0-9149-4d11-87a8-6dd6200249cf"). InnerVolumeSpecName "kube-api-access-dd8dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:59:58.125061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125031    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125061 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125060    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125069    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125077    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70ca3ed0-9149-4d11-87a8-6dd6200249cf-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125086    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd8dl\" (UniqueName: \"kubernetes.io/projected/70ca3ed0-9149-4d11-87a8-6dd6200249cf-kube-api-access-dd8dl\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125095    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.125224 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.125103    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70ca3ed0-9149-4d11-87a8-6dd6200249cf-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:58.633805 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.633774    2575 generic.go:356] "Generic (PLEG): container finished" podID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerID="4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6" exitCode=0
Jul 17 09:59:58.633968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.633820    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"
Jul 17 09:59:58.633968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.633856    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerDied","Data":"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"}
Jul 17 09:59:58.633968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.633911    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj" event={"ID":"70ca3ed0-9149-4d11-87a8-6dd6200249cf","Type":"ContainerDied","Data":"e1c0b35797c4af9b311529f745a98dd19fba3e4b837e90a3eee4d27bd67c2017"}
Jul 17 09:59:58.633968 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.633934    2575 scope.go:117] "RemoveContainer" containerID="4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"
Jul 17 09:59:58.643624 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.643607    2575 scope.go:117] "RemoveContainer" containerID="44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"
Jul 17 09:59:58.651266 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.651251    2575 scope.go:117] "RemoveContainer" containerID="4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"
Jul 17 09:59:58.651530 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:58.651510    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6\": container with ID starting with 4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6 not found: ID does not exist" containerID="4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"
Jul 17 09:59:58.651595 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.651537    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6"} err="failed to get container status \"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6\": rpc error: code = NotFound desc = could not find container \"4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6\": container with ID starting with 4551b16adf0171a82f95c34f08a143c57e24aa333e6c5d16c59ab33019a376b6 not found: ID does not exist"
Jul 17 09:59:58.651595 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.651551    2575 scope.go:117] "RemoveContainer" containerID="44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"
Jul 17 09:59:58.651766 ip-10-0-139-140 kubenswrapper[2575]: E0717 09:59:58.651751    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94\": container with ID starting with 44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94 not found: ID does not exist" containerID="44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"
Jul 17 09:59:58.651803 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.651772    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94"} err="failed to get container status \"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94\": rpc error: code = NotFound desc = could not find container \"44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94\": container with ID starting with 44969b372cd547750f408d57aba81766fbe71f4fc42d152d2fd8938bcfdc0e94 not found: ID does not exist"
Jul 17 09:59:58.665868 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.665848    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:59:58.671121 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:58.671099    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5593c7b6-openshift-default-5c69bd69d7-85glj"]
Jul 17 09:59:59.153281 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.153238    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" path="/var/lib/kubelet/pods/70ca3ed0-9149-4d11-87a8-6dd6200249cf/volumes"
Jul 17 09:59:59.153784 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.153660    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 09:59:59.290877 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.290854    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:59:59.336820 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.336789    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca\") pod \"87957597-5eac-4186-a47c-eaca95614a40\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") "
Jul 17 09:59:59.336956 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.336838    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbmn8\" (UniqueName: \"kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8\") pod \"87957597-5eac-4186-a47c-eaca95614a40\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") "
Jul 17 09:59:59.337048 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.337024    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") pod \"87957597-5eac-4186-a47c-eaca95614a40\" (UID: \"87957597-5eac-4186-a47c-eaca95614a40\") "
Jul 17 09:59:59.337207 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.337188    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87957597-5eac-4186-a47c-eaca95614a40" (UID: "87957597-5eac-4186-a47c-eaca95614a40"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:59.337391 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.337373    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87957597-5eac-4186-a47c-eaca95614a40-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:59.338934 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.338914    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87957597-5eac-4186-a47c-eaca95614a40" (UID: "87957597-5eac-4186-a47c-eaca95614a40"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:59:59.339035 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.339013    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8" (OuterVolumeSpecName: "kube-api-access-nbmn8") pod "87957597-5eac-4186-a47c-eaca95614a40" (UID: "87957597-5eac-4186-a47c-eaca95614a40"). InnerVolumeSpecName "kube-api-access-nbmn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:59:59.438458 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.438364    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbmn8\" (UniqueName: \"kubernetes.io/projected/87957597-5eac-4186-a47c-eaca95614a40-kube-api-access-nbmn8\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:59.438458 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.438403    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87957597-5eac-4186-a47c-eaca95614a40-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 09:59:59.640760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.640676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq" event={"ID":"87957597-5eac-4186-a47c-eaca95614a40","Type":"ContainerDied","Data":"a36361762a3a3f406e550ab6801be80137d00a2888e776713a9e3d1c009cf4d6"}
Jul 17 09:59:59.640760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.640712    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"
Jul 17 09:59:59.640760 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.640747    2575 scope.go:117] "RemoveContainer" containerID="d1327324b7567de051141f43381af3903c20f99724f17441b4a32b63c270e889"
Jul 17 09:59:59.681176 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.681145    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 09:59:59.686023 ip-10-0-139-140 kubenswrapper[2575]: I0717 09:59:59.685993    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-5593c7b6-6f596bb4c7-f99tq"]
Jul 17 10:00:00.144680 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.144639    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"]
Jul 17 10:00:00.145681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145650    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.145681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145682    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145724    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145735    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145786    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145795    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145804    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145812    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145835    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145845    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145867    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.145885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145877    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145894    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145903    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145913    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.145923    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146133    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146148    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146156    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146175    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146184    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146194    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146213    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.146398 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.146229    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ca3ed0-9149-4d11-87a8-6dd6200249cf" containerName="istio-proxy"
Jul 17 10:00:00.151849 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.151817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"
Jul 17 10:00:00.155530 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.155504    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:00:00.157591 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.157566    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"]
Jul 17 10:00:00.163153 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.163132    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"
Jul 17 10:00:00.299058 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.299035    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"]
Jul 17 10:00:00.301240 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:00:00.301210    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0c9901_4908_451d_b874_09d85393c188.slice/crio-c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973 WatchSource:0}: Error finding container c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973: Status 404 returned error can't find the container with id c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973
Jul 17 10:00:00.650175 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.650132    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8" event={"ID":"7d0c9901-4908-451d-b874-09d85393c188","Type":"ContainerStarted","Data":"14ce9a0f4d110657f1c51e47cbb2f6f4a15bf9d6a6c0b1e9bf76cf0fffe9a3f5"}
Jul 17 10:00:00.650385 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.650197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8" event={"ID":"7d0c9901-4908-451d-b874-09d85393c188","Type":"ContainerStarted","Data":"c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973"}
Jul 17 10:00:00.670763 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:00.670715    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8" podStartSLOduration=0.670699478 podStartE2EDuration="670.699478ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:00.668565486 +0000 UTC m=+2514.060558104" watchObservedRunningTime="2026-07-17 10:00:00.670699478 +0000 UTC m=+2514.062692094"
Jul 17 10:00:01.152663 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:01.152627    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87957597-5eac-4186-a47c-eaca95614a40" path="/var/lib/kubelet/pods/87957597-5eac-4186-a47c-eaca95614a40/volumes"
Jul 17 10:00:01.656785 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:01.656749    2575 generic.go:356] "Generic (PLEG): container finished" podID="7d0c9901-4908-451d-b874-09d85393c188" containerID="14ce9a0f4d110657f1c51e47cbb2f6f4a15bf9d6a6c0b1e9bf76cf0fffe9a3f5" exitCode=0
Jul 17 10:00:01.657221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:01.656810    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8" event={"ID":"7d0c9901-4908-451d-b874-09d85393c188","Type":"ContainerDied","Data":"14ce9a0f4d110657f1c51e47cbb2f6f4a15bf9d6a6c0b1e9bf76cf0fffe9a3f5"}
Jul 17 10:00:02.790353 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:02.790330    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"
Jul 17 10:00:03.667203 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:03.667173    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8"
Jul 17 10:00:03.667403 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:03.667206    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-dvrk8" event={"ID":"7d0c9901-4908-451d-b874-09d85393c188","Type":"ContainerDied","Data":"c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973"}
Jul 17 10:00:03.667403 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:03.667249    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c51ea89d7d035ecf37fd768a9d8f510e3b3470ab91e432ae708738ddbffb8973"
Jul 17 10:00:13.959728 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.959689    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960037    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0c9901-4908-451d-b874-09d85393c188" containerName="cleanup"
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960049    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0c9901-4908-451d-b874-09d85393c188" containerName="cleanup"
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960063    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960069    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960079    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:13.960100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960084    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:13.960292 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960142    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0c9901-4908-451d-b874-09d85393c188" containerName="cleanup"
Jul 17 10:00:13.960292 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960153    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87957597-5eac-4186-a47c-eaca95614a40" containerName="maas-api"
Jul 17 10:00:13.960292 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.960161    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fa0878-bfef-4dcb-b0be-cab2cc156402" containerName="revoke-keys"
Jul 17 10:00:13.964702 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.964684    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:13.967652 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.967631    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:00:13.967788 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.967631    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:00:13.973788 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:13.973755    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:14.063374 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.063344    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5\" (UID: \"c949b8e2-2073-4006-95c7-2e8e6c089fe3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:14.164570 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.164537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5\" (UID: \"c949b8e2-2073-4006-95c7-2e8e6c089fe3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:14.165146 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.165127    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5\" (UID: \"c949b8e2-2073-4006-95c7-2e8e6c089fe3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:14.275840 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.275810    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:14.423622 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.423593    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:14.425248 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:00:14.425220    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc949b8e2_2073_4006_95c7_2e8e6c089fe3.slice/crio-ab94bdafbd3bd610d3b3124387421997a6e086dbf0022dc923934dd1272cc5dd WatchSource:0}: Error finding container ab94bdafbd3bd610d3b3124387421997a6e086dbf0022dc923934dd1272cc5dd: Status 404 returned error can't find the container with id ab94bdafbd3bd610d3b3124387421997a6e086dbf0022dc923934dd1272cc5dd
Jul 17 10:00:14.722825 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.722791    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerStarted","Data":"2f8075f194676fce93e8a4a347b394f2032b60cf0641f0f3fed9f60d09e7344e"}
Jul 17 10:00:14.722953 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.722847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerStarted","Data":"ab94bdafbd3bd610d3b3124387421997a6e086dbf0022dc923934dd1272cc5dd"}
Jul 17 10:00:14.746817 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:14.746771    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" podStartSLOduration=1.7467563149999998 podStartE2EDuration="1.746756315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:14.74429333 +0000 UTC m=+2528.136285946" watchObservedRunningTime="2026-07-17 10:00:14.746756315 +0000 UTC m=+2528.138748932"
Jul 17 10:00:15.728354 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:15.728318    2575 generic.go:356] "Generic (PLEG): container finished" podID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerID="2f8075f194676fce93e8a4a347b394f2032b60cf0641f0f3fed9f60d09e7344e" exitCode=6
Jul 17 10:00:15.728707 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:15.728390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerDied","Data":"2f8075f194676fce93e8a4a347b394f2032b60cf0641f0f3fed9f60d09e7344e"}
Jul 17 10:00:15.728707 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:15.728621    2575 scope.go:117] "RemoveContainer" containerID="2f8075f194676fce93e8a4a347b394f2032b60cf0641f0f3fed9f60d09e7344e"
Jul 17 10:00:16.734802 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:16.734763    2575 generic.go:356] "Generic (PLEG): container finished" podID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerID="5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b" exitCode=6
Jul 17 10:00:16.735224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:16.734851    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerDied","Data":"5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b"}
Jul 17 10:00:16.735224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:16.734914    2575 scope.go:117] "RemoveContainer" containerID="2f8075f194676fce93e8a4a347b394f2032b60cf0641f0f3fed9f60d09e7344e"
Jul 17 10:00:16.735224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:16.735034    2575 scope.go:117] "RemoveContainer" containerID="5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b"
Jul 17 10:00:16.735374 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:00:16.735289    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5_odh-ai-gateway-infra(c949b8e2-2073-4006-95c7-2e8e6c089fe3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3"
Jul 17 10:00:17.741567 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:17.741538    2575 scope.go:117] "RemoveContainer" containerID="5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b"
Jul 17 10:00:17.741954 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:00:17.741737    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5_odh-ai-gateway-infra(c949b8e2-2073-4006-95c7-2e8e6c089fe3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3"
Jul 17 10:00:32.147673 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:32.147638    2575 scope.go:117] "RemoveContainer" containerID="5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b"
Jul 17 10:00:32.820383 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:32.820262    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerStarted","Data":"03a5f1d8ea815025bcc473553e96bf3c3093619d34adf8851d376661d1c9930b"}
Jul 17 10:00:33.834008 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:33.833973    2575 generic.go:356] "Generic (PLEG): container finished" podID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerID="03a5f1d8ea815025bcc473553e96bf3c3093619d34adf8851d376661d1c9930b" exitCode=6
Jul 17 10:00:33.834509 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:33.834044    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerDied","Data":"03a5f1d8ea815025bcc473553e96bf3c3093619d34adf8851d376661d1c9930b"}
Jul 17 10:00:33.834509 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:33.834153    2575 scope.go:117] "RemoveContainer" containerID="5d941f6d615d0f93b9f349b58b69c0a66529d6ed948d7ae04f6ea60db166106b"
Jul 17 10:00:33.834509 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:33.834465    2575 scope.go:117] "RemoveContainer" containerID="03a5f1d8ea815025bcc473553e96bf3c3093619d34adf8851d376661d1c9930b"
Jul 17 10:00:33.834686 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:00:33.834669    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5_odh-ai-gateway-infra(c949b8e2-2073-4006-95c7-2e8e6c089fe3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3"
Jul 17 10:00:33.864255 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:33.864234    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:34.975918 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:34.975894    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:35.160811 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.160739    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca\") pod \"c949b8e2-2073-4006-95c7-2e8e6c089fe3\" (UID: \"c949b8e2-2073-4006-95c7-2e8e6c089fe3\") "
Jul 17 10:00:35.161124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.161097    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c949b8e2-2073-4006-95c7-2e8e6c089fe3" (UID: "c949b8e2-2073-4006-95c7-2e8e6c089fe3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:00:35.262170 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.262139    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c949b8e2-2073-4006-95c7-2e8e6c089fe3-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:00:35.846361 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.846326    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"
Jul 17 10:00:35.846361 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.846355    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5" event={"ID":"c949b8e2-2073-4006-95c7-2e8e6c089fe3","Type":"ContainerDied","Data":"ab94bdafbd3bd610d3b3124387421997a6e086dbf0022dc923934dd1272cc5dd"}
Jul 17 10:00:35.846628 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.846414    2575 scope.go:117] "RemoveContainer" containerID="03a5f1d8ea815025bcc473553e96bf3c3093619d34adf8851d376661d1c9930b"
Jul 17 10:00:35.873803 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.873778    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:35.882025 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:35.882003    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-f7ml5"]
Jul 17 10:00:37.153996 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:00:37.153966    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" path="/var/lib/kubelet/pods/c949b8e2-2073-4006-95c7-2e8e6c089fe3/volumes"
Jul 17 10:01:52.428450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428377    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428759    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428772    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428783    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428788    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428798    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428805    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428870    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.428891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.428877    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:01:52.431888 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.431872    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.434765 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.434743    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0d3f0022-serving-cert\""
Jul 17 10:01:52.434906 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.434777    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 10:01:52.435987 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.435974    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 10:01:52.436074 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.436007    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:01:52.458010 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.457973    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:01:52.525474 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.525434    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.525614 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.525561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.525614 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.525593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2c9\" (UniqueName: \"kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.626591 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.626566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.626760 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.626624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.626760 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.626646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2c9\" (UniqueName: \"kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.626760 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:01:52.626724    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-serving-cert: secret "maas-api-e2e-mt-0d3f0022-serving-cert" not found
Jul 17 10:01:52.626929 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:01:52.626796    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls podName:b1eb9257-723b-4c68-9243-e2b31375ec16 nodeName:}" failed. No retries permitted until 2026-07-17 10:01:53.126773739 +0000 UTC m=+2626.518766337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls") pod "maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" (UID: "b1eb9257-723b-4c68-9243-e2b31375ec16") : secret "maas-api-e2e-mt-0d3f0022-serving-cert" not found
Jul 17 10:01:52.627264 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.627244    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:52.641975 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:52.641953    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2c9\" (UniqueName: \"kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:53.131789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:53.131756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:53.134327 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:53.134279    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") pod \"maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:53.342655 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:53.342620    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:53.484799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:53.484772    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:01:53.487552 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:01:53.487520    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1eb9257_723b_4c68_9243_e2b31375ec16.slice/crio-e8dc8a0e120ac10353fff2e30e85e8093cfa1449e582877343c332563c9c50f1 WatchSource:0}: Error finding container e8dc8a0e120ac10353fff2e30e85e8093cfa1449e582877343c332563c9c50f1: Status 404 returned error can't find the container with id e8dc8a0e120ac10353fff2e30e85e8093cfa1449e582877343c332563c9c50f1
Jul 17 10:01:53.489272 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:53.489253    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:01:54.224638 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:54.224601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" event={"ID":"b1eb9257-723b-4c68-9243-e2b31375ec16","Type":"ContainerStarted","Data":"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"}
Jul 17 10:01:54.224805 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:54.224681    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" event={"ID":"b1eb9257-723b-4c68-9243-e2b31375ec16","Type":"ContainerStarted","Data":"e8dc8a0e120ac10353fff2e30e85e8093cfa1449e582877343c332563c9c50f1"}
Jul 17 10:01:54.224805 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:54.224795    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:01:54.248207 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:01:54.248162    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" podStartSLOduration=1.832162645 podStartE2EDuration="2.248148975s" totalImagesPullingTime="415.98633ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:01:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:01:54.247945607 +0000 UTC m=+2627.639938225" watchObservedRunningTime="2026-07-17 10:01:54.248148975 +0000 UTC m=+2627.640141617"
Jul 17 10:02:00.234677 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:00.234644    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:02:15.561769 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.561736    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:02:15.562246 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.561988    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" podUID="b1eb9257-723b-4c68-9243-e2b31375ec16" containerName="maas-api" containerID="cri-o://987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d" gracePeriod=30
Jul 17 10:02:15.796909 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.796886    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:02:15.932014 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.931930    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca\") pod \"b1eb9257-723b-4c68-9243-e2b31375ec16\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") "
Jul 17 10:02:15.932014 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.931970    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") pod \"b1eb9257-723b-4c68-9243-e2b31375ec16\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") "
Jul 17 10:02:15.932232 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.932043    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf2c9\" (UniqueName: \"kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9\") pod \"b1eb9257-723b-4c68-9243-e2b31375ec16\" (UID: \"b1eb9257-723b-4c68-9243-e2b31375ec16\") "
Jul 17 10:02:15.932319 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.932278    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1eb9257-723b-4c68-9243-e2b31375ec16" (UID: "b1eb9257-723b-4c68-9243-e2b31375ec16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:15.934051 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.934019    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1eb9257-723b-4c68-9243-e2b31375ec16" (UID: "b1eb9257-723b-4c68-9243-e2b31375ec16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:02:15.934051 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:15.934033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9" (OuterVolumeSpecName: "kube-api-access-qf2c9") pod "b1eb9257-723b-4c68-9243-e2b31375ec16" (UID: "b1eb9257-723b-4c68-9243-e2b31375ec16"). InnerVolumeSpecName "kube-api-access-qf2c9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:02:16.032954 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.032932    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1eb9257-723b-4c68-9243-e2b31375ec16-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:02:16.032954 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.032952    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1eb9257-723b-4c68-9243-e2b31375ec16-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:02:16.033090 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.032961    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf2c9\" (UniqueName: \"kubernetes.io/projected/b1eb9257-723b-4c68-9243-e2b31375ec16-kube-api-access-qf2c9\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:02:16.332107 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.332066    2575 generic.go:356] "Generic (PLEG): container finished" podID="b1eb9257-723b-4c68-9243-e2b31375ec16" containerID="987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d" exitCode=0
Jul 17 10:02:16.332107 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.332106    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"
Jul 17 10:02:16.332343 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.332129    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" event={"ID":"b1eb9257-723b-4c68-9243-e2b31375ec16","Type":"ContainerDied","Data":"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"}
Jul 17 10:02:16.332343 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.332184    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl" event={"ID":"b1eb9257-723b-4c68-9243-e2b31375ec16","Type":"ContainerDied","Data":"e8dc8a0e120ac10353fff2e30e85e8093cfa1449e582877343c332563c9c50f1"}
Jul 17 10:02:16.332343 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.332207    2575 scope.go:117] "RemoveContainer" containerID="987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"
Jul 17 10:02:16.341121 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.341101    2575 scope.go:117] "RemoveContainer" containerID="987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"
Jul 17 10:02:16.341426 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:02:16.341409    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d\": container with ID starting with 987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d not found: ID does not exist" containerID="987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"
Jul 17 10:02:16.341496 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.341434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d"} err="failed to get container status \"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d\": rpc error: code = NotFound desc = could not find container \"987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d\": container with ID starting with 987e81120fb35a01d1daa83d626560149379e16576085b2e2d9590c3e5f8f04d not found: ID does not exist"
Jul 17 10:02:16.357624 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.357598    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:02:16.360470 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:16.360448    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d3f0022-7bd4b58c9d-bmmcl"]
Jul 17 10:02:17.152739 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:17.152705    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1eb9257-723b-4c68-9243-e2b31375ec16" path="/var/lib/kubelet/pods/b1eb9257-723b-4c68-9243-e2b31375ec16/volumes"
Jul 17 10:02:54.153739 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.153711    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:02:54.154210 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.154043    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1eb9257-723b-4c68-9243-e2b31375ec16" containerName="maas-api"
Jul 17 10:02:54.154210 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.154054    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1eb9257-723b-4c68-9243-e2b31375ec16" containerName="maas-api"
Jul 17 10:02:54.154210 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.154120    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1eb9257-723b-4c68-9243-e2b31375ec16" containerName="maas-api"
Jul 17 10:02:54.154210 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.154131    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c949b8e2-2073-4006-95c7-2e8e6c089fe3" containerName="revoke-keys"
Jul 17 10:02:54.156759 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.156741    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:02:54.159897 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.159875    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:02:54.160070 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.160054    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:02:54.167614 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.167589    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:02:54.257691 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.257662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2\" (UID: \"343680b3-1e88-4871-a0dd-dbad54940ff1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:02:54.358659 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.358605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2\" (UID: \"343680b3-1e88-4871-a0dd-dbad54940ff1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:02:54.359250 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.359232    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2\" (UID: \"343680b3-1e88-4871-a0dd-dbad54940ff1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:02:54.468658 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.468568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:02:54.600220 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:54.600038    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:02:54.603066 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:02:54.603035    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343680b3_1e88_4871_a0dd_dbad54940ff1.slice/crio-ccdd49abd3c5878c12e6febd35930fa0e37a80447f78f3871bc00c9a68e6523d WatchSource:0}: Error finding container ccdd49abd3c5878c12e6febd35930fa0e37a80447f78f3871bc00c9a68e6523d: Status 404 returned error can't find the container with id ccdd49abd3c5878c12e6febd35930fa0e37a80447f78f3871bc00c9a68e6523d
Jul 17 10:02:55.536478 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.536395    2575 generic.go:356] "Generic (PLEG): container finished" podID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerID="dead7ade91998f196960224dae844820b3bde169a857b56dabcf891e8d30352a" exitCode=6
Jul 17 10:02:55.536839 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.536476    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerDied","Data":"dead7ade91998f196960224dae844820b3bde169a857b56dabcf891e8d30352a"}
Jul 17 10:02:55.536839 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.536553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerStarted","Data":"ccdd49abd3c5878c12e6febd35930fa0e37a80447f78f3871bc00c9a68e6523d"}
Jul 17 10:02:55.536839 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.536621    2575 scope.go:117] "RemoveContainer" containerID="dead7ade91998f196960224dae844820b3bde169a857b56dabcf891e8d30352a"
Jul 17 10:02:55.828757 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.828688    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:02:55.832066 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.832049    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:55.835486 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.835423    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-43c677b2-serving-cert\""
Jul 17 10:02:55.835486 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.835450    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 10:02:55.835486 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.835466    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 10:02:55.843886 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.843855    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:02:55.971809 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.971775    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:55.971991 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.971824    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:55.971991 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:55.971890    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx5nr\" (UniqueName: \"kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.072726 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.072689    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.072890 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.072739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.072890 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.072781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx5nr\" (UniqueName: \"kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.073536 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.073510    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.075111 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.075087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.084341 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.084271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx5nr\" (UniqueName: \"kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr\") pod \"maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.143291 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.143263    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:56.276351 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.276326    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:02:56.279239 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:02:56.279178    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac89088_deca_4760_ae6d_137ea2e3f6b1.slice/crio-fc1cfeff5d808cfa1bd36abc79ec47675bb37a96ff4be59c5fd0a1ca0b458696 WatchSource:0}: Error finding container fc1cfeff5d808cfa1bd36abc79ec47675bb37a96ff4be59c5fd0a1ca0b458696: Status 404 returned error can't find the container with id fc1cfeff5d808cfa1bd36abc79ec47675bb37a96ff4be59c5fd0a1ca0b458696
Jul 17 10:02:56.543477 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.543442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" event={"ID":"5ac89088-deca-4760-ae6d-137ea2e3f6b1","Type":"ContainerStarted","Data":"fc1cfeff5d808cfa1bd36abc79ec47675bb37a96ff4be59c5fd0a1ca0b458696"}
Jul 17 10:02:56.545447 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.545414    2575 generic.go:356] "Generic (PLEG): container finished" podID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerID="adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a" exitCode=6
Jul 17 10:02:56.545599 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.545469    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerDied","Data":"adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a"}
Jul 17 10:02:56.545599 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.545527    2575 scope.go:117] "RemoveContainer" containerID="dead7ade91998f196960224dae844820b3bde169a857b56dabcf891e8d30352a"
Jul 17 10:02:56.545744 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:56.545727    2575 scope.go:117] "RemoveContainer" containerID="adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a"
Jul 17 10:02:56.545967 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:02:56.545945    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2_odh-ai-gateway-infra(343680b3-1e88-4871-a0dd-dbad54940ff1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1"
Jul 17 10:02:57.551675 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:57.551640    2575 scope.go:117] "RemoveContainer" containerID="adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a"
Jul 17 10:02:57.552106 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:02:57.551871    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2_odh-ai-gateway-infra(343680b3-1e88-4871-a0dd-dbad54940ff1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1"
Jul 17 10:02:57.552850 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:57.552826    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" event={"ID":"5ac89088-deca-4760-ae6d-137ea2e3f6b1","Type":"ContainerStarted","Data":"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"}
Jul 17 10:02:57.552987 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:57.552873    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:02:57.584774 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:02:57.584722    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" podStartSLOduration=2.159767836 podStartE2EDuration="2.584703567s" totalImagesPullingTime="424.935731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:02:57.584233198 +0000 UTC m=+2690.976225819" watchObservedRunningTime="2026-07-17 10:02:57.584703567 +0000 UTC m=+2690.976696184"
Jul 17 10:03:03.563422 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:03.563340    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:03:07.299143 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:07.299110    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:03:07.302490 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:07.302468    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:03:07.304448 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:07.304423    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:03:07.307912 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:07.307892    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:03:08.147678 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:08.147648    2575 scope.go:117] "RemoveContainer" containerID="adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a"
Jul 17 10:03:08.606996 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:08.606968    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerStarted","Data":"b394e6a0bbb03c3a87f9ada76be776396c4d915aeafc7200090b569d6a457a32"}
Jul 17 10:03:08.624727 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:08.624683    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" podStartSLOduration=14.62466962 podStartE2EDuration="14.62466962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:02:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:08.623155188 +0000 UTC m=+2702.015147806" watchObservedRunningTime="2026-07-17 10:03:08.62466962 +0000 UTC m=+2702.016662237"
Jul 17 10:03:09.612865 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:09.612829    2575 generic.go:356] "Generic (PLEG): container finished" podID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerID="b394e6a0bbb03c3a87f9ada76be776396c4d915aeafc7200090b569d6a457a32" exitCode=6
Jul 17 10:03:09.613373 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:09.612901    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerDied","Data":"b394e6a0bbb03c3a87f9ada76be776396c4d915aeafc7200090b569d6a457a32"}
Jul 17 10:03:09.613373 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:09.612968    2575 scope.go:117] "RemoveContainer" containerID="adb1b7c7c721c435376311de30f8bec460f2806059d8cdfb6faba2188322e64a"
Jul 17 10:03:09.613373 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:09.613262    2575 scope.go:117] "RemoveContainer" containerID="b394e6a0bbb03c3a87f9ada76be776396c4d915aeafc7200090b569d6a457a32"
Jul 17 10:03:09.613571 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:03:09.613529    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2_odh-ai-gateway-infra(343680b3-1e88-4871-a0dd-dbad54940ff1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1"
Jul 17 10:03:09.640812 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:09.640787    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:03:10.748688 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:10.748666    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:03:10.901253 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:10.901154    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca\") pod \"343680b3-1e88-4871-a0dd-dbad54940ff1\" (UID: \"343680b3-1e88-4871-a0dd-dbad54940ff1\") "
Jul 17 10:03:10.901539 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:10.901516    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "343680b3-1e88-4871-a0dd-dbad54940ff1" (UID: "343680b3-1e88-4871-a0dd-dbad54940ff1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:11.004328 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.002692    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/343680b3-1e88-4871-a0dd-dbad54940ff1-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:03:11.625620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.625593    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"
Jul 17 10:03:11.625620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.625612    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2" event={"ID":"343680b3-1e88-4871-a0dd-dbad54940ff1","Type":"ContainerDied","Data":"ccdd49abd3c5878c12e6febd35930fa0e37a80447f78f3871bc00c9a68e6523d"}
Jul 17 10:03:11.625851 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.625668    2575 scope.go:117] "RemoveContainer" containerID="b394e6a0bbb03c3a87f9ada76be776396c4d915aeafc7200090b569d6a457a32"
Jul 17 10:03:11.646050 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.646026    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:03:11.651909 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:11.651890    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-p8bn2"]
Jul 17 10:03:13.153983 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:13.153948    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" path="/var/lib/kubelet/pods/343680b3-1e88-4871-a0dd-dbad54940ff1/volumes"
Jul 17 10:03:20.679469 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679431    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679810    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679823    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679846    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679852    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679911    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679919    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.679935 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.679926    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:03:20.682913 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.682897    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.687589 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.687564    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-43c677b2\"/\"kube-root-ca.crt\""
Jul 17 10:03:20.687743 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.687693    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-43c677b2\"/\"e2e-lifecycle-model-43c677b2-kserve-self-signed-certs\""
Jul 17 10:03:20.687743 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.687719    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-43c677b2\"/\"default-dockercfg-d6nbt\""
Jul 17 10:03:20.687874 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.687796    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-43c677b2\"/\"openshift-service-ca.crt\""
Jul 17 10:03:20.694065 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.694047    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:03:20.780554 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780515    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.780684 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.780684 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.780775 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.780775 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhs8m\" (UniqueName: \"kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.780840 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.780788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881454 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881479    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881516    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhs8m\" (UniqueName: \"kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881750 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881870 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881847    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881958 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881937    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.881997 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.881967    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.883733 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.883709    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.884104 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.884074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.889901 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.889880    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhs8m\" (UniqueName: \"kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m\") pod \"e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") " pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:20.992085 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:20.992053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:21.125937 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:21.125908    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:03:21.127810 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:03:21.127784    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37622fac_8f7b_4315_8ee9_b9fbf4af1e3e.slice/crio-407765978cc1fdd2e7d08aa11b71ea425267f3afacf02003f46cb94f0f8b5638 WatchSource:0}: Error finding container 407765978cc1fdd2e7d08aa11b71ea425267f3afacf02003f46cb94f0f8b5638: Status 404 returned error can't find the container with id 407765978cc1fdd2e7d08aa11b71ea425267f3afacf02003f46cb94f0f8b5638
Jul 17 10:03:21.675673 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:21.675636    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" event={"ID":"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e","Type":"ContainerStarted","Data":"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"}
Jul 17 10:03:21.675673 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:21.675676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" event={"ID":"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e","Type":"ContainerStarted","Data":"407765978cc1fdd2e7d08aa11b71ea425267f3afacf02003f46cb94f0f8b5638"}
Jul 17 10:03:21.706824 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:21.706778    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" podStartSLOduration=1.706762764 podStartE2EDuration="1.706762764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:21.698416945 +0000 UTC m=+2715.090409560" watchObservedRunningTime="2026-07-17 10:03:21.706762764 +0000 UTC m=+2715.098755381"
Jul 17 10:03:30.992605 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:30.992569    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:30.993068 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:30.992620    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:31.004481 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:31.004459    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:31.736268 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:31.736238    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:03:53.382262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.382226    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:03:53.382739 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.382451    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerName="maas-api" containerID="cri-o://6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a" gracePeriod=30
Jul 17 10:03:53.627594 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.627569    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:03:53.764351 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.764287    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls\") pod \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") "
Jul 17 10:03:53.764498 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.764428    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx5nr\" (UniqueName: \"kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr\") pod \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") "
Jul 17 10:03:53.764498 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.764451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca\") pod \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\" (UID: \"5ac89088-deca-4760-ae6d-137ea2e3f6b1\") "
Jul 17 10:03:53.764852 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.764825    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ac89088-deca-4760-ae6d-137ea2e3f6b1" (UID: "5ac89088-deca-4760-ae6d-137ea2e3f6b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:53.766329 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.766287    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ac89088-deca-4760-ae6d-137ea2e3f6b1" (UID: "5ac89088-deca-4760-ae6d-137ea2e3f6b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:03:53.766466 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.766447    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr" (OuterVolumeSpecName: "kube-api-access-kx5nr") pod "5ac89088-deca-4760-ae6d-137ea2e3f6b1" (UID: "5ac89088-deca-4760-ae6d-137ea2e3f6b1"). InnerVolumeSpecName "kube-api-access-kx5nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:03:53.832323 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.832284    2575 generic.go:356] "Generic (PLEG): container finished" podID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerID="6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a" exitCode=0
Jul 17 10:03:53.832429 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.832345    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"
Jul 17 10:03:53.832429 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.832365    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" event={"ID":"5ac89088-deca-4760-ae6d-137ea2e3f6b1","Type":"ContainerDied","Data":"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"}
Jul 17 10:03:53.832429 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.832419    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" event={"ID":"5ac89088-deca-4760-ae6d-137ea2e3f6b1","Type":"ContainerDied","Data":"fc1cfeff5d808cfa1bd36abc79ec47675bb37a96ff4be59c5fd0a1ca0b458696"}
Jul 17 10:03:53.832553 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.832444    2575 scope.go:117] "RemoveContainer" containerID="6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"
Jul 17 10:03:53.841528 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.841512    2575 scope.go:117] "RemoveContainer" containerID="6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"
Jul 17 10:03:53.841783 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:03:53.841765    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a\": container with ID starting with 6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a not found: ID does not exist" containerID="6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"
Jul 17 10:03:53.841835 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.841790    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a"} err="failed to get container status \"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a\": rpc error: code = NotFound desc = could not find container \"6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a\": container with ID starting with 6b5a888f98a4ed072b16b7e266eb1daf9a12f0a921501e03383d6180b63b8d5a not found: ID does not exist"
Jul 17 10:03:53.854144 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.854120    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:03:53.855789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.855771    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw"]
Jul 17 10:03:53.865858 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.865840    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac89088-deca-4760-ae6d-137ea2e3f6b1-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:03:53.865919 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.865860    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx5nr\" (UniqueName: \"kubernetes.io/projected/5ac89088-deca-4760-ae6d-137ea2e3f6b1-kube-api-access-kx5nr\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:03:53.865919 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:53.865870    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac89088-deca-4760-ae6d-137ea2e3f6b1-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:03:55.153287 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:55.153250    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" path="/var/lib/kubelet/pods/5ac89088-deca-4760-ae6d-137ea2e3f6b1/volumes"
Jul 17 10:03:56.559538 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:03:56.559481    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-43c677b2-7f6bb95cd9-65tcw" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.50:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 17 10:04:03.348743 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:03.348708    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:04:18.370091 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.370046    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" podUID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" containerName="main" containerID="cri-o://d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a" gracePeriod=45
Jul 17 10:04:18.619638 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.619614    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:04:18.692169 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692097    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692165    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692206    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692241    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692267    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692330    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhs8m\" (UniqueName: \"kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m\") pod \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\" (UID: \"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e\") "
Jul 17 10:04:18.692606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692467    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:04:18.692606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692490    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home" (OuterVolumeSpecName: "home") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:04:18.692606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692497    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:04:18.692768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692686    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tmp-dir\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.692768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692704    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-home\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.692768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.692716    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-model-cache\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.694563 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.694527    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm" (OuterVolumeSpecName: "dshm") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:04:18.694673 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.694601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m" (OuterVolumeSpecName: "kube-api-access-jhs8m") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "kube-api-access-jhs8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:04:18.694724 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.694712    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" (UID: "37622fac-8f7b-4315-8ee9-b9fbf4af1e3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:04:18.794014 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.793983    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-dshm\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.794014 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.794010    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhs8m\" (UniqueName: \"kubernetes.io/projected/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-kube-api-access-jhs8m\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.794014 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.794020    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e-tls-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:04:18.955547 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.955462    2575 generic.go:356] "Generic (PLEG): container finished" podID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" containerID="d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a" exitCode=0
Jul 17 10:04:18.955547 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.955504    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"
Jul 17 10:04:18.955716 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.955558    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" event={"ID":"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e","Type":"ContainerDied","Data":"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"}
Jul 17 10:04:18.955716 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.955596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz" event={"ID":"37622fac-8f7b-4315-8ee9-b9fbf4af1e3e","Type":"ContainerDied","Data":"407765978cc1fdd2e7d08aa11b71ea425267f3afacf02003f46cb94f0f8b5638"}
Jul 17 10:04:18.955716 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.955615    2575 scope.go:117] "RemoveContainer" containerID="d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"
Jul 17 10:04:18.965003 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.964989    2575 scope.go:117] "RemoveContainer" containerID="d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"
Jul 17 10:04:18.965233 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:04:18.965216    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a\": container with ID starting with d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a not found: ID does not exist" containerID="d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"
Jul 17 10:04:18.965273 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.965241    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a"} err="failed to get container status \"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a\": rpc error: code = NotFound desc = could not find container \"d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a\": container with ID starting with d0a5b228f23d74d896ca2efb8be3875aec595f4fa478042a396ad14f1032b19a not found: ID does not exist"
Jul 17 10:04:18.978081 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.978054    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:04:18.982373 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:18.982348    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-43c677b2/e2e-lifecycle-model-43c677b2-kserve-5679dd97ff-5k4rz"]
Jul 17 10:04:19.152878 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:19.152849    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" path="/var/lib/kubelet/pods/37622fac-8f7b-4315-8ee9-b9fbf4af1e3e/volumes"
Jul 17 10:04:53.319768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.319688    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320025    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" containerName="main"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320035    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" containerName="main"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320044    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320050    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="343680b3-1e88-4871-a0dd-dbad54940ff1" containerName="revoke-keys"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320069    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerName="maas-api"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320075    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerName="maas-api"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320132    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="37622fac-8f7b-4315-8ee9-b9fbf4af1e3e" containerName="main"
Jul 17 10:04:53.320502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.320143    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ac89088-deca-4760-ae6d-137ea2e3f6b1" containerName="maas-api"
Jul 17 10:04:53.322815 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.322797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:04:53.326854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.326833    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:04:53.326854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.326845    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:04:53.348350 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.348325    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:04:53.486850 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.486817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99\" (UID: \"3339b80f-9e19-42ae-8b0e-77d8d9cb6788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:04:53.588174 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.588103    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99\" (UID: \"3339b80f-9e19-42ae-8b0e-77d8d9cb6788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:04:53.588714 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.588694    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99\" (UID: \"3339b80f-9e19-42ae-8b0e-77d8d9cb6788\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:04:53.632746 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.632718    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:04:53.793100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:53.793076    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:04:53.795373 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:04:53.795347    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3339b80f_9e19_42ae_8b0e_77d8d9cb6788.slice/crio-ce77678a87cacb410e3046840aa53b7b3f9b488668d1a220caec963d847b3cfa WatchSource:0}: Error finding container ce77678a87cacb410e3046840aa53b7b3f9b488668d1a220caec963d847b3cfa: Status 404 returned error can't find the container with id ce77678a87cacb410e3046840aa53b7b3f9b488668d1a220caec963d847b3cfa
Jul 17 10:04:54.125866 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:54.125836    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerStarted","Data":"cc62eca4c85b29990749b219e77bde00a5189ecaca11df6c21c1bc509d944ec6"}
Jul 17 10:04:54.125991 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:54.125890    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerStarted","Data":"ce77678a87cacb410e3046840aa53b7b3f9b488668d1a220caec963d847b3cfa"}
Jul 17 10:04:54.159071 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:54.159029    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" podStartSLOduration=1.159013915 podStartE2EDuration="1.159013915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:04:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:04:54.154200944 +0000 UTC m=+2807.546193561" watchObservedRunningTime="2026-07-17 10:04:54.159013915 +0000 UTC m=+2807.551006564"
Jul 17 10:04:55.132078 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:55.132046    2575 generic.go:356] "Generic (PLEG): container finished" podID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerID="cc62eca4c85b29990749b219e77bde00a5189ecaca11df6c21c1bc509d944ec6" exitCode=6
Jul 17 10:04:55.132501 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:55.132129    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerDied","Data":"cc62eca4c85b29990749b219e77bde00a5189ecaca11df6c21c1bc509d944ec6"}
Jul 17 10:04:55.132501 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:55.132270    2575 scope.go:117] "RemoveContainer" containerID="cc62eca4c85b29990749b219e77bde00a5189ecaca11df6c21c1bc509d944ec6"
Jul 17 10:04:56.139099 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:56.139068    2575 generic.go:356] "Generic (PLEG): container finished" podID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerID="747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8" exitCode=6
Jul 17 10:04:56.139521 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:56.139157    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerDied","Data":"747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8"}
Jul 17 10:04:56.139521 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:56.139221    2575 scope.go:117] "RemoveContainer" containerID="cc62eca4c85b29990749b219e77bde00a5189ecaca11df6c21c1bc509d944ec6"
Jul 17 10:04:56.139521 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:56.139434    2575 scope.go:117] "RemoveContainer" containerID="747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8"
Jul 17 10:04:56.139660 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:04:56.139637    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99_odh-ai-gateway-infra(3339b80f-9e19-42ae-8b0e-77d8d9cb6788)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788"
Jul 17 10:04:57.146441 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:04:57.146407    2575 scope.go:117] "RemoveContainer" containerID="747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8"
Jul 17 10:04:57.146917 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:04:57.146691    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99_odh-ai-gateway-infra(3339b80f-9e19-42ae-8b0e-77d8d9cb6788)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788"
Jul 17 10:05:10.147247 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:10.147218    2575 scope.go:117] "RemoveContainer" containerID="747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8"
Jul 17 10:05:11.211252 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:11.211176    2575 generic.go:356] "Generic (PLEG): container finished" podID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerID="ec397b6638f5374d336d4a2b8529e33c915e7153ff6bbe2623b0b9d1c0db3771" exitCode=6
Jul 17 10:05:11.211643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:11.211248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerDied","Data":"ec397b6638f5374d336d4a2b8529e33c915e7153ff6bbe2623b0b9d1c0db3771"}
Jul 17 10:05:11.211643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:11.211323    2575 scope.go:117] "RemoveContainer" containerID="747d0ace64ace73dd15ed0fed441e3468052b29b2e5350bf30d1e5595335fde8"
Jul 17 10:05:11.211643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:11.211563    2575 scope.go:117] "RemoveContainer" containerID="ec397b6638f5374d336d4a2b8529e33c915e7153ff6bbe2623b0b9d1c0db3771"
Jul 17 10:05:11.211801 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:05:11.211782    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99_odh-ai-gateway-infra(3339b80f-9e19-42ae-8b0e-77d8d9cb6788)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788"
Jul 17 10:05:12.255387 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:12.255352    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:05:12.383831 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:12.383807    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:05:12.443242 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:12.443214    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca\") pod \"3339b80f-9e19-42ae-8b0e-77d8d9cb6788\" (UID: \"3339b80f-9e19-42ae-8b0e-77d8d9cb6788\") "
Jul 17 10:05:12.443579 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:12.443555    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3339b80f-9e19-42ae-8b0e-77d8d9cb6788" (UID: "3339b80f-9e19-42ae-8b0e-77d8d9cb6788"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:12.543888 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:12.543822    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3339b80f-9e19-42ae-8b0e-77d8d9cb6788-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:05:13.222899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:13.222870    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99" event={"ID":"3339b80f-9e19-42ae-8b0e-77d8d9cb6788","Type":"ContainerDied","Data":"ce77678a87cacb410e3046840aa53b7b3f9b488668d1a220caec963d847b3cfa"}
Jul 17 10:05:13.223060 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:13.222922    2575 scope.go:117] "RemoveContainer" containerID="ec397b6638f5374d336d4a2b8529e33c915e7153ff6bbe2623b0b9d1c0db3771"
Jul 17 10:05:13.223060 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:13.222882    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"
Jul 17 10:05:13.240053 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:13.240030    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:05:13.247154 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:13.247127    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-gpn99"]
Jul 17 10:05:15.153126 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:15.153096    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" path="/var/lib/kubelet/pods/3339b80f-9e19-42ae-8b0e-77d8d9cb6788/volumes"
Jul 17 10:05:49.505959 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.505924    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506312    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506326    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506342    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506347    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506357    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506362    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506420    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.506465 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.506434    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:49.509569 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.509552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.522408 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.522386    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fe50f0-openshift-default-dockercfg-xv84w\""
Jul 17 10:05:49.542592 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.542566    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:05:49.663828 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.663792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664005 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.663830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664005 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.663909    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664005 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.663944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664005 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.663998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664261 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.664026    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664261 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.664071    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664261 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.664114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f62qb\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.664261 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.664151    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765360 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765269    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f62qb\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765360 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765325    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765553 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765553 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765393    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765665 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765665 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765653    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765776 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765776 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765879 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765879 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.765879 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765804    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.766036 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.765956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.766175 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.766149    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.766518 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.766494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.767901 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.767875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.768043 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.768029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.775411 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.775375    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f62qb\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.776091 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.776068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token\") pod \"e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") " pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.820081 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.820057    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:49.990271 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.990242    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:05:49.991783 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:05:49.991751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8166dd_5d21_481a_b9ab_95d9d318daa3.slice/crio-ae20d9bbe76dd5f051c1c7e78977eb091ba53d4a73810ac2add23de6220e1736 WatchSource:0}: Error finding container ae20d9bbe76dd5f051c1c7e78977eb091ba53d4a73810ac2add23de6220e1736: Status 404 returned error can't find the container with id ae20d9bbe76dd5f051c1c7e78977eb091ba53d4a73810ac2add23de6220e1736
Jul 17 10:05:49.994342 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.994272    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:05:49.994413 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.994394    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:05:49.994463 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:49.994444    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:05:50.394836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:50.394738    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerStarted","Data":"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"}
Jul 17 10:05:50.394836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:50.394793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerStarted","Data":"ae20d9bbe76dd5f051c1c7e78977eb091ba53d4a73810ac2add23de6220e1736"}
Jul 17 10:05:50.427661 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:50.427620    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podStartSLOduration=1.427604336 podStartE2EDuration="1.427604336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:05:50.426886477 +0000 UTC m=+2863.818879094" watchObservedRunningTime="2026-07-17 10:05:50.427604336 +0000 UTC m=+2863.819596952"
Jul 17 10:05:50.820716 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:50.820686    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:51.825494 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:51.825453    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:52.405265 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.405237    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:52.406310 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.406267    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:05:52.605693 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.605659    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:05:52.606156 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.606139    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3339b80f-9e19-42ae-8b0e-77d8d9cb6788" containerName="revoke-keys"
Jul 17 10:05:52.609294 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.609278    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.612266 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.612237    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:05:52.612266 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.612242    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 10:05:52.612836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.612823    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 10:05:52.613066 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.613048    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fe50f0-serving-cert\""
Jul 17 10:05:52.622116 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.622095    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:05:52.691330 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.691247    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.691330 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.691273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.691490 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.691421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcwtn\" (UniqueName: \"kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.792869 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.792840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcwtn\" (UniqueName: \"kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.793001 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.792892    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.793001 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.792908    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.793001 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:05:52.792986    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-serving-cert: secret "maas-api-e2e-inf-a-fe50f0-serving-cert" not found
Jul 17 10:05:52.793130 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:05:52.793063    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls podName:ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a nodeName:}" failed. No retries permitted until 2026-07-17 10:05:53.293042146 +0000 UTC m=+2866.685034742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls") pod "maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" (UID: "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a") : secret "maas-api-e2e-inf-a-fe50f0-serving-cert" not found
Jul 17 10:05:52.793546 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.793529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:52.804085 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:52.804057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcwtn\" (UniqueName: \"kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:53.298183 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:53.298151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:53.300603 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:53.300583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") pod \"maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:53.520071 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:53.520034    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:53.653880 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:53.653853    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:05:53.655706 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:05:53.655677    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3039bb_5c2b_4c7f_8b9b_2725cf4ea40a.slice/crio-4aeeb2787a5b72e5492848e83b8caeba404986714c3576f6c0a0e7063d226408 WatchSource:0}: Error finding container 4aeeb2787a5b72e5492848e83b8caeba404986714c3576f6c0a0e7063d226408: Status 404 returned error can't find the container with id 4aeeb2787a5b72e5492848e83b8caeba404986714c3576f6c0a0e7063d226408
Jul 17 10:05:54.416135 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:54.416094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" event={"ID":"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a","Type":"ContainerStarted","Data":"4aeeb2787a5b72e5492848e83b8caeba404986714c3576f6c0a0e7063d226408"}
Jul 17 10:05:55.422229 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:55.422191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" event={"ID":"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a","Type":"ContainerStarted","Data":"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"}
Jul 17 10:05:55.422712 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:55.422336    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:05:55.442305 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:05:55.442259    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" podStartSLOduration=2.541097985 podStartE2EDuration="3.442245041s" totalImagesPullingTime="901.147056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:05:55.441651451 +0000 UTC m=+2868.833644063" watchObservedRunningTime="2026-07-17 10:05:55.442245041 +0000 UTC m=+2868.834237657"
Jul 17 10:06:00.532184 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.532096    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:06:00.536063 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.536041    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.539030 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.539009    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-787694-serving-cert\""
Jul 17 10:06:00.557360 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.557333    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:06:00.558742 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.558717    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.558854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.558753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxdl6\" (UniqueName: \"kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.558854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.558834    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.659487 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.659446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.659487 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.659486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdl6\" (UniqueName: \"kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.659817 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.659535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.659817 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:06:00.659618    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-serving-cert: secret "maas-api-e2e-inf-b-787694-serving-cert" not found
Jul 17 10:06:00.659817 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:06:00.659705    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls podName:fdc420dd-134c-4d47-a347-8b63bf94e6ac nodeName:}" failed. No retries permitted until 2026-07-17 10:06:01.159680913 +0000 UTC m=+2874.551673547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls") pod "maas-api-e2e-inf-b-787694-555947f569-c6tlr" (UID: "fdc420dd-134c-4d47-a347-8b63bf94e6ac") : secret "maas-api-e2e-inf-b-787694-serving-cert" not found
Jul 17 10:06:00.660255 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.660229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:00.685099 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:00.685068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxdl6\" (UniqueName: \"kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:01.163561 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:01.163531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:01.165936 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:01.165914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-787694-555947f569-c6tlr\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:01.432131 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:01.432054    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:06:01.448111 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:01.448080    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:01.864343 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:01.864284    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:06:01.868114 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:06:01.868086    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc420dd_134c_4d47_a347_8b63bf94e6ac.slice/crio-cb2471a6ee31d0aca75cc77ab9a3895d6eb485eeb174e23dc41bbe49fb19dcf4 WatchSource:0}: Error finding container cb2471a6ee31d0aca75cc77ab9a3895d6eb485eeb174e23dc41bbe49fb19dcf4: Status 404 returned error can't find the container with id cb2471a6ee31d0aca75cc77ab9a3895d6eb485eeb174e23dc41bbe49fb19dcf4
Jul 17 10:06:02.456849 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:02.456806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" event={"ID":"fdc420dd-134c-4d47-a347-8b63bf94e6ac","Type":"ContainerStarted","Data":"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"}
Jul 17 10:06:02.456974 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:02.456872    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:02.456974 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:02.456892    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" event={"ID":"fdc420dd-134c-4d47-a347-8b63bf94e6ac","Type":"ContainerStarted","Data":"cb2471a6ee31d0aca75cc77ab9a3895d6eb485eeb174e23dc41bbe49fb19dcf4"}
Jul 17 10:06:02.498520 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:02.498455    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" podStartSLOduration=2.023608367 podStartE2EDuration="2.498431916s" totalImagesPullingTime="474.823549ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:02.495692575 +0000 UTC m=+2875.887685195" watchObservedRunningTime="2026-07-17 10:06:02.498431916 +0000 UTC m=+2875.890424533"
Jul 17 10:06:08.031001 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.030032    2575 patch_prober.go:28] interesting pod/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:54506->10.133.0.53:15021: read: connection reset by peer" start-of-body=
Jul 17 10:06:08.031001 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.030108    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:54506->10.133.0.53:15021: read: connection reset by peer"
Jul 17 10:06:08.467192 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.467117    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:06:08.488681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.488651    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerID="f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad" exitCode=0
Jul 17 10:06:08.488826 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.488721    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerDied","Data":"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"}
Jul 17 10:06:08.489110 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.489096    2575 scope.go:117] "RemoveContainer" containerID="f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"
Jul 17 10:06:08.490259 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.490232    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:06:08.490350 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.490333    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:06:08.490397 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:08.490378    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:06:09.500359 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:09.499837    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerStarted","Data":"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"}
Jul 17 10:06:09.821048 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:09.820948    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:06:09.821212 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:09.821195    2575 patch_prober.go:28] interesting pod/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 17 10:06:09.821286 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:09.821264    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 17 10:06:10.820927 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:10.820893    2575 patch_prober.go:28] interesting pod/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 17 10:06:10.821326 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:10.820958    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 17 10:06:11.820469 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:11.820432    2575 patch_prober.go:28] interesting pod/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 17 10:06:11.820652 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:11.820489    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 17 10:06:12.820820 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:12.820785    2575 patch_prober.go:28] interesting pod/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 17 10:06:12.821215 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:12.820840    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 17 10:06:13.824755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:13.824725    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:06:13.825107 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:13.825018    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:06:13.825766 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:06:13.825747    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:07:15.734734 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:15.734697    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:07:15.735219 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:15.734892    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" podUID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" containerName="maas-api" containerID="cri-o://f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e" gracePeriod=30
Jul 17 10:07:15.976982 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:15.976958    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:07:16.044124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.044027    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca\") pod \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") "
Jul 17 10:07:16.044124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.044092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") pod \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") "
Jul 17 10:07:16.044376 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.044152    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcwtn\" (UniqueName: \"kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn\") pod \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\" (UID: \"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a\") "
Jul 17 10:07:16.044495 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.044469    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" (UID: "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:16.046265 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.046236    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" (UID: "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:07:16.046407 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.046337    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn" (OuterVolumeSpecName: "kube-api-access-vcwtn") pod "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" (UID: "ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a"). InnerVolumeSpecName "kube-api-access-vcwtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:07:16.145789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.145744    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:16.145789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.145787    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:16.145789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.145798    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcwtn\" (UniqueName: \"kubernetes.io/projected/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a-kube-api-access-vcwtn\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:16.830097 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.830063    2575 generic.go:356] "Generic (PLEG): container finished" podID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" containerID="f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e" exitCode=0
Jul 17 10:07:16.830514 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.830113    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"
Jul 17 10:07:16.830514 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.830103    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" event={"ID":"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a","Type":"ContainerDied","Data":"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"}
Jul 17 10:07:16.830514 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.830240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx" event={"ID":"ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a","Type":"ContainerDied","Data":"4aeeb2787a5b72e5492848e83b8caeba404986714c3576f6c0a0e7063d226408"}
Jul 17 10:07:16.830514 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.830269    2575 scope.go:117] "RemoveContainer" containerID="f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"
Jul 17 10:07:16.840090 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.840070    2575 scope.go:117] "RemoveContainer" containerID="f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"
Jul 17 10:07:16.840356 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:16.840336    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e\": container with ID starting with f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e not found: ID does not exist" containerID="f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"
Jul 17 10:07:16.840427 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.840365    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e"} err="failed to get container status \"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e\": rpc error: code = NotFound desc = could not find container \"f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e\": container with ID starting with f96c28ee2cdda43fa905fc2f8414a0b4c59b5d5d05a0bbed405a0939d09e224e not found: ID does not exist"
Jul 17 10:07:16.853840 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.853817    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:07:16.855566 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:16.855546    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fe50f0-6ff6c8b455-kzqwx"]
Jul 17 10:07:17.152993 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:17.152915    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" path="/var/lib/kubelet/pods/ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a/volumes"
Jul 17 10:07:31.034084 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:31.034004    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:07:31.034537 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:31.034230    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy" containerID="cri-o://803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05" gracePeriod=30
Jul 17 10:07:36.292225 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.292198    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:07:36.425498 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425398    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425498 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425451    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425498 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425480    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425564    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425625    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425655    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f62qb\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425711    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425731    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:07:36.425799 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.426114 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.425824    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket\") pod \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\" (UID: \"7a8166dd-5d21-481a-b9ab-95d9d318daa3\") "
Jul 17 10:07:36.426114 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426030    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:36.426114 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426094    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:07:36.426269 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426223    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:07:36.426450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426425    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.426450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426452    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.426641 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426461    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:07:36.426641 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426468    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.426641 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.426484    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.427876 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.427847    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:07:36.427992 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.427908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:07:36.428134 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.428114    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:07:36.428294 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.428271    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb" (OuterVolumeSpecName: "kube-api-access-f62qb") pod "7a8166dd-5d21-481a-b9ab-95d9d318daa3" (UID: "7a8166dd-5d21-481a-b9ab-95d9d318daa3"). InnerVolumeSpecName "kube-api-access-f62qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:07:36.527364 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.527334    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.527364 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.527357    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.527364 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.527366    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.527567 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.527375    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a8166dd-5d21-481a-b9ab-95d9d318daa3-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.527567 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.527383    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f62qb\" (UniqueName: \"kubernetes.io/projected/7a8166dd-5d21-481a-b9ab-95d9d318daa3-kube-api-access-f62qb\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:36.929879 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.929845    2575 generic.go:356] "Generic (PLEG): container finished" podID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerID="803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05" exitCode=0
Jul 17 10:07:36.930054 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.929891    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"
Jul 17 10:07:36.930054 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.929912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerDied","Data":"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"}
Jul 17 10:07:36.930054 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.929956    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng" event={"ID":"7a8166dd-5d21-481a-b9ab-95d9d318daa3","Type":"ContainerDied","Data":"ae20d9bbe76dd5f051c1c7e78977eb091ba53d4a73810ac2add23de6220e1736"}
Jul 17 10:07:36.930054 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.929978    2575 scope.go:117] "RemoveContainer" containerID="803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"
Jul 17 10:07:36.939191 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.939173    2575 scope.go:117] "RemoveContainer" containerID="f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"
Jul 17 10:07:36.947815 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.947795    2575 scope.go:117] "RemoveContainer" containerID="803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"
Jul 17 10:07:36.948074 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:36.948054    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05\": container with ID starting with 803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05 not found: ID does not exist" containerID="803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"
Jul 17 10:07:36.948124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.948084    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05"} err="failed to get container status \"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05\": rpc error: code = NotFound desc = could not find container \"803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05\": container with ID starting with 803ebab951f0b0b7b6596ad2bb02c695d6b7208347f152789d9ad54b7e039c05 not found: ID does not exist"
Jul 17 10:07:36.948124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.948103    2575 scope.go:117] "RemoveContainer" containerID="f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"
Jul 17 10:07:36.948600 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:36.948575    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad\": container with ID starting with f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad not found: ID does not exist" containerID="f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"
Jul 17 10:07:36.948748 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.948725    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad"} err="failed to get container status \"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad\": rpc error: code = NotFound desc = could not find container \"f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad\": container with ID starting with f72720629ec4402eea8e2658bad7465514b78e63e691a5d4d8f340ddb654e5ad not found: ID does not exist"
Jul 17 10:07:36.953002 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.952976    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:07:36.957187 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:36.957164    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fe50f0-openshift-default-5dcfcdf74f-j6zng"]
Jul 17 10:07:37.154225 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:37.154193    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" path="/var/lib/kubelet/pods/7a8166dd-5d21-481a-b9ab-95d9d318daa3/volumes"
Jul 17 10:07:42.171968 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.171932    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:07:42.172438 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.172131    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" podUID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" containerName="maas-api" containerID="cri-o://5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22" gracePeriod=30
Jul 17 10:07:42.414879 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.414858    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:07:42.477282 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.477257    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca\") pod \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") "
Jul 17 10:07:42.477447 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.477318    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxdl6\" (UniqueName: \"kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6\") pod \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") "
Jul 17 10:07:42.477447 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.477347    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") pod \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\" (UID: \"fdc420dd-134c-4d47-a347-8b63bf94e6ac\") "
Jul 17 10:07:42.477679 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.477654    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdc420dd-134c-4d47-a347-8b63bf94e6ac" (UID: "fdc420dd-134c-4d47-a347-8b63bf94e6ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:42.490213 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.490183    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6" (OuterVolumeSpecName: "kube-api-access-vxdl6") pod "fdc420dd-134c-4d47-a347-8b63bf94e6ac" (UID: "fdc420dd-134c-4d47-a347-8b63bf94e6ac"). InnerVolumeSpecName "kube-api-access-vxdl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:07:42.490389 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.490365    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdc420dd-134c-4d47-a347-8b63bf94e6ac" (UID: "fdc420dd-134c-4d47-a347-8b63bf94e6ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:07:42.578740 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.578713    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc420dd-134c-4d47-a347-8b63bf94e6ac-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:42.578740 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.578738    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxdl6\" (UniqueName: \"kubernetes.io/projected/fdc420dd-134c-4d47-a347-8b63bf94e6ac-kube-api-access-vxdl6\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:42.578891 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.578749    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdc420dd-134c-4d47-a347-8b63bf94e6ac-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:07:42.962362 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.962268    2575 generic.go:356] "Generic (PLEG): container finished" podID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" containerID="5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22" exitCode=0
Jul 17 10:07:42.962362 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.962328    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"
Jul 17 10:07:42.962362 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.962349    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" event={"ID":"fdc420dd-134c-4d47-a347-8b63bf94e6ac","Type":"ContainerDied","Data":"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"}
Jul 17 10:07:42.962579 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.962401    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr" event={"ID":"fdc420dd-134c-4d47-a347-8b63bf94e6ac","Type":"ContainerDied","Data":"cb2471a6ee31d0aca75cc77ab9a3895d6eb485eeb174e23dc41bbe49fb19dcf4"}
Jul 17 10:07:42.962579 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.962424    2575 scope.go:117] "RemoveContainer" containerID="5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"
Jul 17 10:07:42.976416 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.976398    2575 scope.go:117] "RemoveContainer" containerID="5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"
Jul 17 10:07:42.976682 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:42.976661    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22\": container with ID starting with 5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22 not found: ID does not exist" containerID="5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"
Jul 17 10:07:42.976734 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.976691    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22"} err="failed to get container status \"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22\": rpc error: code = NotFound desc = could not find container \"5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22\": container with ID starting with 5a505125761e84f98284f2bfe00102bd10d84cd4ca70d4ea30e75185e9f6db22 not found: ID does not exist"
Jul 17 10:07:42.990929 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.990907    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:07:42.993356 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:42.993334    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-787694-555947f569-c6tlr"]
Jul 17 10:07:43.153288 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:43.153257    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" path="/var/lib/kubelet/pods/fdc420dd-134c-4d47-a347-8b63bf94e6ac/volumes"
Jul 17 10:07:55.873934 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.873898    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:07:55.874453 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874428    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:55.874453 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874452    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874468    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" containerName="maas-api"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874477    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" containerName="maas-api"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874497    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" containerName="maas-api"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874507    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" containerName="maas-api"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874531    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:55.874606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874540    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:55.874790 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874627    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:55.874790 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874643    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac3039bb-5c2b-4c7f-8b9b-2725cf4ea40a" containerName="maas-api"
Jul 17 10:07:55.874790 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.874652    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc420dd-134c-4d47-a347-8b63bf94e6ac" containerName="maas-api"
Jul 17 10:07:55.877872 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.877853    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:07:55.881011 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.880990    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:07:55.881551 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.881531    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:07:55.887894 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:55.887870    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:07:56.000688 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:56.000643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7\" (UID: \"44e9e107-347c-40e1-9632-4792675d5730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:07:56.101932 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:56.101904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7\" (UID: \"44e9e107-347c-40e1-9632-4792675d5730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:07:56.102556 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:56.102537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7\" (UID: \"44e9e107-347c-40e1-9632-4792675d5730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:07:56.190314 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:56.190228    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:07:56.318387 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:56.318350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:07:56.320646 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:07:56.320620    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e9e107_347c_40e1_9632_4792675d5730.slice/crio-33093f425af313ce882c4e1eae36b32bfd9d41e8d17f4ff7d78a7877225daba7 WatchSource:0}: Error finding container 33093f425af313ce882c4e1eae36b32bfd9d41e8d17f4ff7d78a7877225daba7: Status 404 returned error can't find the container with id 33093f425af313ce882c4e1eae36b32bfd9d41e8d17f4ff7d78a7877225daba7
Jul 17 10:07:57.030774 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:57.030738    2575 generic.go:356] "Generic (PLEG): container finished" podID="44e9e107-347c-40e1-9632-4792675d5730" containerID="fca685aac6d75ea4201719888eb720ab1ebc8e31ae06e9cd8e72b1a55c65f1ee" exitCode=6
Jul 17 10:07:57.031154 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:57.030797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" event={"ID":"44e9e107-347c-40e1-9632-4792675d5730","Type":"ContainerDied","Data":"fca685aac6d75ea4201719888eb720ab1ebc8e31ae06e9cd8e72b1a55c65f1ee"}
Jul 17 10:07:57.031154 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:57.030835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" event={"ID":"44e9e107-347c-40e1-9632-4792675d5730","Type":"ContainerStarted","Data":"33093f425af313ce882c4e1eae36b32bfd9d41e8d17f4ff7d78a7877225daba7"}
Jul 17 10:07:57.031154 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:57.031016    2575 scope.go:117] "RemoveContainer" containerID="fca685aac6d75ea4201719888eb720ab1ebc8e31ae06e9cd8e72b1a55c65f1ee"
Jul 17 10:07:58.036961 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.036928    2575 generic.go:356] "Generic (PLEG): container finished" podID="44e9e107-347c-40e1-9632-4792675d5730" containerID="ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c" exitCode=6
Jul 17 10:07:58.037375 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.036973    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" event={"ID":"44e9e107-347c-40e1-9632-4792675d5730","Type":"ContainerDied","Data":"ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c"}
Jul 17 10:07:58.037375 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.037037    2575 scope.go:117] "RemoveContainer" containerID="fca685aac6d75ea4201719888eb720ab1ebc8e31ae06e9cd8e72b1a55c65f1ee"
Jul 17 10:07:58.037375 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.037250    2575 scope.go:117] "RemoveContainer" containerID="ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c"
Jul 17 10:07:58.037539 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:58.037511    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7_odh-ai-gateway-infra(44e9e107-347c-40e1-9632-4792675d5730)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" podUID="44e9e107-347c-40e1-9632-4792675d5730"
Jul 17 10:07:58.489997 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.489958    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:07:58.490486 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.490468    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a8166dd-5d21-481a-b9ab-95d9d318daa3" containerName="istio-proxy"
Jul 17 10:07:58.494532 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.494511    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.498841 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.498819    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d7262f-openshift-default-dockercfg-t9x9h\""
Jul 17 10:07:58.523779 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.523882 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523789    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.523882 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523833    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.523882 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d8nx\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.523989 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523943    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.524024 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.523998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.524058 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.524029    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.524108 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.524095    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.524145 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.524126    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.580729 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.580699    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:07:58.624832 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.624802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.624969 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.624838    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.624969 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.624958    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625088 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.624997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625088 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625018    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625088 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d8nx\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625238 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625102    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625238 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625238 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625441 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625281    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625560    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625678 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625735 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.625994 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.625975    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.627227 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.627204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.627360 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.627345    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.636428 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.636409    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d8nx\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.637184 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.637161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token\") pod \"e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") " pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.805184 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.805101    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:07:58.977502 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.977461    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:07:58.980355 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:07:58.980319    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7ab5fa_c05b_40dd_9e5b_0cfa29dbc5c8.slice/crio-4759bebcbccad90b5371090cba6e85adb6b379a944a01a1dca364675aa89b308 WatchSource:0}: Error finding container 4759bebcbccad90b5371090cba6e85adb6b379a944a01a1dca364675aa89b308: Status 404 returned error can't find the container with id 4759bebcbccad90b5371090cba6e85adb6b379a944a01a1dca364675aa89b308
Jul 17 10:07:58.982462 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.982426    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:07:58.982548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.982491    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:07:58.982548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:58.982521    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:07:59.042703 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:59.042670    2575 scope.go:117] "RemoveContainer" containerID="ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c"
Jul 17 10:07:59.043006 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:07:59.042899    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7_odh-ai-gateway-infra(44e9e107-347c-40e1-9632-4792675d5730)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" podUID="44e9e107-347c-40e1-9632-4792675d5730"
Jul 17 10:07:59.044054 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:59.044030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerStarted","Data":"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"}
Jul 17 10:07:59.044133 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:59.044077    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerStarted","Data":"4759bebcbccad90b5371090cba6e85adb6b379a944a01a1dca364675aa89b308"}
Jul 17 10:07:59.117600 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:59.117491    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podStartSLOduration=1.117474397 podStartE2EDuration="1.117474397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:59.113448292 +0000 UTC m=+2992.505440931" watchObservedRunningTime="2026-07-17 10:07:59.117474397 +0000 UTC m=+2992.509466995"
Jul 17 10:07:59.805422 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:07:59.805388    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:00.811232 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:00.811197    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:01.053653 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.053627    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:01.054657 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.054636    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:01.377059 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.377028    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:01.380630 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.380609    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.383798 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.383778    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d7262f-serving-cert\""
Jul 17 10:08:01.384556 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.384538    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 10:08:01.384641 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.384537    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 10:08:01.392650 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.392628    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:01.444354 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.444286    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgts8\" (UniqueName: \"kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.444533 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.444358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.444533 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.444497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.545141 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.545104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.545344 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.545151    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgts8\" (UniqueName: \"kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.545344 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.545172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.545344 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:01.545252    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-serving-cert: secret "maas-api-e2e-api-a-d7262f-serving-cert" not found
Jul 17 10:08:01.545344 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:01.545332    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls podName:1d05e1bf-b1bf-4584-9756-128563dba320 nodeName:}" failed. No retries permitted until 2026-07-17 10:08:02.045311115 +0000 UTC m=+2995.437303728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls") pod "maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" (UID: "1d05e1bf-b1bf-4584-9756-128563dba320") : secret "maas-api-e2e-api-a-d7262f-serving-cert" not found
Jul 17 10:08:01.545818 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.545800    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:01.558679 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:01.558645    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgts8\" (UniqueName: \"kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:02.049102 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:02.049063    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:02.051574 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:02.051554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") pod \"maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:02.291210 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:02.291169    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:02.433286 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:02.433261    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:02.435181 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:08:02.435143    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d05e1bf_b1bf_4584_9756_128563dba320.slice/crio-3e2c6b7387743c9d3b7a299f111d85407faa8679710fa7c716e3ea41af2094d4 WatchSource:0}: Error finding container 3e2c6b7387743c9d3b7a299f111d85407faa8679710fa7c716e3ea41af2094d4: Status 404 returned error can't find the container with id 3e2c6b7387743c9d3b7a299f111d85407faa8679710fa7c716e3ea41af2094d4
Jul 17 10:08:02.436641 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:02.436622    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:08:03.064959 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:03.064924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" event={"ID":"1d05e1bf-b1bf-4584-9756-128563dba320","Type":"ContainerStarted","Data":"3e2c6b7387743c9d3b7a299f111d85407faa8679710fa7c716e3ea41af2094d4"}
Jul 17 10:08:04.071450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:04.071418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" event={"ID":"1d05e1bf-b1bf-4584-9756-128563dba320","Type":"ContainerStarted","Data":"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"}
Jul 17 10:08:04.071851 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:04.071523    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:04.092546 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:04.092496    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" podStartSLOduration=2.443372667 podStartE2EDuration="3.09248111s" totalImagesPullingTime="649.108443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:04.090477069 +0000 UTC m=+2997.482469688" watchObservedRunningTime="2026-07-17 10:08:04.09248111 +0000 UTC m=+2997.484473726"
Jul 17 10:08:07.329621 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:07.329587    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:08:07.336313 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:07.336281    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:08:07.338079 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:07.338044    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:08:07.343822 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:07.343806    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:08:10.090624 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.090596    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:10.248628 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.248594    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:08:10.253579 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.253561    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.256684 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.256663    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-439eab-serving-cert\""
Jul 17 10:08:10.270681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.270655    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:08:10.326824 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.326782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.326824 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.326829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.327041 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.326898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpq6l\" (UniqueName: \"kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.428248 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.428156    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.428248 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.428198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.428518 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.428347    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpq6l\" (UniqueName: \"kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.428518 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:10.428355    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-serving-cert: secret "maas-api-e2e-api-b-439eab-serving-cert" not found
Jul 17 10:08:10.428634 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:10.428547    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls podName:2f042011-b64a-4346-b37b-ffc5acb0dac2 nodeName:}" failed. No retries permitted until 2026-07-17 10:08:10.928520707 +0000 UTC m=+3004.320513306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls") pod "maas-api-e2e-api-b-439eab-88d498947-2rmcx" (UID: "2f042011-b64a-4346-b37b-ffc5acb0dac2") : secret "maas-api-e2e-api-b-439eab-serving-cert" not found
Jul 17 10:08:10.428932 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.428913    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.451893 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.451869    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpq6l\" (UniqueName: \"kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.933123 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.933085    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:10.935532 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:10.935511    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") pod \"maas-api-e2e-api-b-439eab-88d498947-2rmcx\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:11.164080 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:11.164045    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:11.373767 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:08:11.373717    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f042011_b64a_4346_b37b_ffc5acb0dac2.slice/crio-b3d734c1daba11d7f8a2afef2aeb31aa03f43bf4fa40d60a2c3abec158c8bf82 WatchSource:0}: Error finding container b3d734c1daba11d7f8a2afef2aeb31aa03f43bf4fa40d60a2c3abec158c8bf82: Status 404 returned error can't find the container with id b3d734c1daba11d7f8a2afef2aeb31aa03f43bf4fa40d60a2c3abec158c8bf82
Jul 17 10:08:11.389088 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:11.375621    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:08:12.120248 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:12.120213    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" event={"ID":"2f042011-b64a-4346-b37b-ffc5acb0dac2","Type":"ContainerStarted","Data":"291e6d2b5c39a0461dafb4b93be3dead9b9263fa9c6327721a118a78d4cbab7b"}
Jul 17 10:08:12.120435 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:12.120274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" event={"ID":"2f042011-b64a-4346-b37b-ffc5acb0dac2","Type":"ContainerStarted","Data":"b3d734c1daba11d7f8a2afef2aeb31aa03f43bf4fa40d60a2c3abec158c8bf82"}
Jul 17 10:08:12.120435 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:12.120327    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:12.142725 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:12.142679    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" podStartSLOduration=1.529042531 podStartE2EDuration="2.142664561s" totalImagesPullingTime="613.62203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:12.14205016 +0000 UTC m=+3005.534042770" watchObservedRunningTime="2026-07-17 10:08:12.142664561 +0000 UTC m=+3005.534657178"
Jul 17 10:08:12.147626 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:12.147605    2575 scope.go:117] "RemoveContainer" containerID="ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c"
Jul 17 10:08:13.128926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:13.128896    2575 generic.go:356] "Generic (PLEG): container finished" podID="44e9e107-347c-40e1-9632-4792675d5730" containerID="c033b699b4855c367c4cff74e9cafb90eb37e6dcce0516bddeea2ac9502f9723" exitCode=6
Jul 17 10:08:13.129329 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:13.128978    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" event={"ID":"44e9e107-347c-40e1-9632-4792675d5730","Type":"ContainerDied","Data":"c033b699b4855c367c4cff74e9cafb90eb37e6dcce0516bddeea2ac9502f9723"}
Jul 17 10:08:13.129329 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:13.129058    2575 scope.go:117] "RemoveContainer" containerID="ab6bab2898a145b45ffe812b1c94bf1cf54144383c726c4d162422272c67685c"
Jul 17 10:08:13.129459 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:13.129413    2575 scope.go:117] "RemoveContainer" containerID="c033b699b4855c367c4cff74e9cafb90eb37e6dcce0516bddeea2ac9502f9723"
Jul 17 10:08:13.129670 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:13.129643    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7_odh-ai-gateway-infra(44e9e107-347c-40e1-9632-4792675d5730)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" podUID="44e9e107-347c-40e1-9632-4792675d5730"
Jul 17 10:08:14.168684 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:14.168650    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:08:14.301554 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:14.301521    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:08:14.368374 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:14.368342    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca\") pod \"44e9e107-347c-40e1-9632-4792675d5730\" (UID: \"44e9e107-347c-40e1-9632-4792675d5730\") "
Jul 17 10:08:14.380938 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:14.368675    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44e9e107-347c-40e1-9632-4792675d5730" (UID: "44e9e107-347c-40e1-9632-4792675d5730"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:14.380938 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:14.368788    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44e9e107-347c-40e1-9632-4792675d5730-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:15.142668 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.142636    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"
Jul 17 10:08:15.142859 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.142642    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7" event={"ID":"44e9e107-347c-40e1-9632-4792675d5730","Type":"ContainerDied","Data":"33093f425af313ce882c4e1eae36b32bfd9d41e8d17f4ff7d78a7877225daba7"}
Jul 17 10:08:15.142859 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.142775    2575 scope.go:117] "RemoveContainer" containerID="c033b699b4855c367c4cff74e9cafb90eb37e6dcce0516bddeea2ac9502f9723"
Jul 17 10:08:15.144255 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.144231    2575 generic.go:356] "Generic (PLEG): container finished" podID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerID="a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441" exitCode=0
Jul 17 10:08:15.144412 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.144313    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerDied","Data":"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"}
Jul 17 10:08:15.144726 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.144708    2575 scope.go:117] "RemoveContainer" containerID="a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"
Jul 17 10:08:15.145739 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.145710    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:08:15.145866 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.145780    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:08:15.145866 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.145812    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:08:15.200325 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.198329    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:08:15.209110 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:15.209080    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-n6dn7"]
Jul 17 10:08:16.053593 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:16.053557    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:16.152864 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:16.152835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerStarted","Data":"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"}
Jul 17 10:08:16.805631 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:16.805591    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:16.806847 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:16.806817    2575 patch_prober.go:28] interesting pod/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 17 10:08:16.806963 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:16.806867    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 17 10:08:17.155092 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:17.155011    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e9e107-347c-40e1-9632-4792675d5730" path="/var/lib/kubelet/pods/44e9e107-347c-40e1-9632-4792675d5730/volumes"
Jul 17 10:08:17.805971 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:17.805929    2575 patch_prober.go:28] interesting pod/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 17 10:08:17.806387 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:17.805989    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 17 10:08:18.135017 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:18.134939    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:08:18.806085 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:18.806042    2575 patch_prober.go:28] interesting pod/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 17 10:08:18.806509 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:18.806105    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 17 10:08:19.806167 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:19.806130    2575 patch_prober.go:28] interesting pod/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 17 10:08:19.806613 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:19.806183    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 17 10:08:20.809563 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:20.809530    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:20.809952 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:20.809898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:20.810563 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:20.810544    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:33.479279 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.479243    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:33.479789 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.479476    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" podUID="1d05e1bf-b1bf-4584-9756-128563dba320" containerName="maas-api" containerID="cri-o://bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542" gracePeriod=30
Jul 17 10:08:33.731586 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.731526    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:33.856316 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.856265    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") pod \"1d05e1bf-b1bf-4584-9756-128563dba320\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") "
Jul 17 10:08:33.856470 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.856387    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgts8\" (UniqueName: \"kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8\") pod \"1d05e1bf-b1bf-4584-9756-128563dba320\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") "
Jul 17 10:08:33.856470 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.856427    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca\") pod \"1d05e1bf-b1bf-4584-9756-128563dba320\" (UID: \"1d05e1bf-b1bf-4584-9756-128563dba320\") "
Jul 17 10:08:33.856836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.856810    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d05e1bf-b1bf-4584-9756-128563dba320" (UID: "1d05e1bf-b1bf-4584-9756-128563dba320"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:33.858348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.858326    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d05e1bf-b1bf-4584-9756-128563dba320" (UID: "1d05e1bf-b1bf-4584-9756-128563dba320"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:08:33.858425 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.858405    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8" (OuterVolumeSpecName: "kube-api-access-qgts8") pod "1d05e1bf-b1bf-4584-9756-128563dba320" (UID: "1d05e1bf-b1bf-4584-9756-128563dba320"). InnerVolumeSpecName "kube-api-access-qgts8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:08:33.957174 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.957145    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgts8\" (UniqueName: \"kubernetes.io/projected/1d05e1bf-b1bf-4584-9756-128563dba320-kube-api-access-qgts8\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:33.957174 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.957170    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d05e1bf-b1bf-4584-9756-128563dba320-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:33.957337 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:33.957182    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d05e1bf-b1bf-4584-9756-128563dba320-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:34.242055 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.242028    2575 generic.go:356] "Generic (PLEG): container finished" podID="1d05e1bf-b1bf-4584-9756-128563dba320" containerID="bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542" exitCode=0
Jul 17 10:08:34.242224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.242078    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"
Jul 17 10:08:34.242224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.242117    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" event={"ID":"1d05e1bf-b1bf-4584-9756-128563dba320","Type":"ContainerDied","Data":"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"}
Jul 17 10:08:34.242224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.242174    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494" event={"ID":"1d05e1bf-b1bf-4584-9756-128563dba320","Type":"ContainerDied","Data":"3e2c6b7387743c9d3b7a299f111d85407faa8679710fa7c716e3ea41af2094d4"}
Jul 17 10:08:34.242224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.242196    2575 scope.go:117] "RemoveContainer" containerID="bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"
Jul 17 10:08:34.251183 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.251166    2575 scope.go:117] "RemoveContainer" containerID="bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"
Jul 17 10:08:34.251455 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:34.251438    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542\": container with ID starting with bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542 not found: ID does not exist" containerID="bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"
Jul 17 10:08:34.251516 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.251462    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542"} err="failed to get container status \"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542\": rpc error: code = NotFound desc = could not find container \"bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542\": container with ID starting with bf073147862aa9179d3033b0ada0e9c5ce6e4bffd6989064b15d455fe661b542 not found: ID does not exist"
Jul 17 10:08:34.267604 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.267575    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:34.270838 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:34.270814    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d7262f-5dd9c6b477-rs494"]
Jul 17 10:08:35.153543 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:35.153506    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d05e1bf-b1bf-4584-9756-128563dba320" path="/var/lib/kubelet/pods/1d05e1bf-b1bf-4584-9756-128563dba320/volumes"
Jul 17 10:08:48.881208 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:48.881172    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:08:48.881840 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:48.881482    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" containerID="cri-o://b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003" gracePeriod=30
Jul 17 10:08:50.812170 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:50.812125    2575 patch_prober.go:28] interesting pod/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 10:08:50.812653 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:50.812211    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 10:08:54.128534 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.128510    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:54.130666 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130647    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130722 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130688    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130758 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130724    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130758 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130749    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130827 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130790    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d8nx\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130858 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130823    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130909 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130891    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130965 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.130965 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130915    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:08:54.131075 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.130970    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert\") pod \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\" (UID: \"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8\") "
Jul 17 10:08:54.131075 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131039    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:08:54.131181 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131146    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:08:54.131355 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131333    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.131457 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131358    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.131457 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131374    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.131457 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131423    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:54.131617 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.131602    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:08:54.132939 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.132917    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:08:54.133135 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.133117    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:08:54.133443 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.133411    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:08:54.133443 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.133423    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx" (OuterVolumeSpecName: "kube-api-access-4d8nx") pod "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" (UID: "5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8"). InnerVolumeSpecName "kube-api-access-4d8nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:08:54.232354 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232325    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.232354 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232350    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.232548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232364    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d8nx\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-kube-api-access-4d8nx\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.232548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232376    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.232548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232388    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.232548 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.232398    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.346137 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.346099    2575 generic.go:356] "Generic (PLEG): container finished" podID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerID="b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003" exitCode=0
Jul 17 10:08:54.346335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.346180    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"
Jul 17 10:08:54.346335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.346186    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerDied","Data":"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"}
Jul 17 10:08:54.346335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.346272    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58" event={"ID":"5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8","Type":"ContainerDied","Data":"4759bebcbccad90b5371090cba6e85adb6b379a944a01a1dca364675aa89b308"}
Jul 17 10:08:54.346335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.346320    2575 scope.go:117] "RemoveContainer" containerID="b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"
Jul 17 10:08:54.356170 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.356154    2575 scope.go:117] "RemoveContainer" containerID="a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"
Jul 17 10:08:54.364757 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.364739    2575 scope.go:117] "RemoveContainer" containerID="b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"
Jul 17 10:08:54.365002 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:54.364983    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003\": container with ID starting with b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003 not found: ID does not exist" containerID="b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"
Jul 17 10:08:54.365070 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.365013    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003"} err="failed to get container status \"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003\": rpc error: code = NotFound desc = could not find container \"b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003\": container with ID starting with b6f100a4ed8a1acc21dd51db77cee31bdf27b00dcfb8b790ec4280a46a0a8003 not found: ID does not exist"
Jul 17 10:08:54.365070 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.365038    2575 scope.go:117] "RemoveContainer" containerID="a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"
Jul 17 10:08:54.365341 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:08:54.365323    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441\": container with ID starting with a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441 not found: ID does not exist" containerID="a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"
Jul 17 10:08:54.365429 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.365347    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441"} err="failed to get container status \"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441\": rpc error: code = NotFound desc = could not find container \"a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441\": container with ID starting with a4cf7ae20a496fe27d27507581fda50479ed9313ce72f56f5098a2ad08ea4441 not found: ID does not exist"
Jul 17 10:08:54.370628 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.370602    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:08:54.375954 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:54.375933    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d7262f-openshift-default-5ccf9784c7-4dz58"]
Jul 17 10:08:55.153645 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:08:55.153613    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" path="/var/lib/kubelet/pods/5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8/volumes"
Jul 17 10:09:00.245446 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.245342    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:09:00.245980 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.245682    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" podUID="2f042011-b64a-4346-b37b-ffc5acb0dac2" containerName="maas-api" containerID="cri-o://291e6d2b5c39a0461dafb4b93be3dead9b9263fa9c6327721a118a78d4cbab7b" gracePeriod=30
Jul 17 10:09:00.378663 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.378636    2575 generic.go:356] "Generic (PLEG): container finished" podID="2f042011-b64a-4346-b37b-ffc5acb0dac2" containerID="291e6d2b5c39a0461dafb4b93be3dead9b9263fa9c6327721a118a78d4cbab7b" exitCode=0
Jul 17 10:09:00.378798 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.378666    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" event={"ID":"2f042011-b64a-4346-b37b-ffc5acb0dac2","Type":"ContainerDied","Data":"291e6d2b5c39a0461dafb4b93be3dead9b9263fa9c6327721a118a78d4cbab7b"}
Jul 17 10:09:00.494291 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.494269    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:09:00.588106 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.588029    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") pod \"2f042011-b64a-4346-b37b-ffc5acb0dac2\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") "
Jul 17 10:09:00.588106 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.588078    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca\") pod \"2f042011-b64a-4346-b37b-ffc5acb0dac2\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") "
Jul 17 10:09:00.588315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.588116    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpq6l\" (UniqueName: \"kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l\") pod \"2f042011-b64a-4346-b37b-ffc5acb0dac2\" (UID: \"2f042011-b64a-4346-b37b-ffc5acb0dac2\") "
Jul 17 10:09:00.588504 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.588475    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f042011-b64a-4346-b37b-ffc5acb0dac2" (UID: "2f042011-b64a-4346-b37b-ffc5acb0dac2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:00.590183 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.590150    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f042011-b64a-4346-b37b-ffc5acb0dac2" (UID: "2f042011-b64a-4346-b37b-ffc5acb0dac2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:09:00.590183 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.590172    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l" (OuterVolumeSpecName: "kube-api-access-zpq6l") pod "2f042011-b64a-4346-b37b-ffc5acb0dac2" (UID: "2f042011-b64a-4346-b37b-ffc5acb0dac2"). InnerVolumeSpecName "kube-api-access-zpq6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:09:00.689594 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.689568    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f042011-b64a-4346-b37b-ffc5acb0dac2-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:09:00.689594 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.689593    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f042011-b64a-4346-b37b-ffc5acb0dac2-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:09:00.689750 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:00.689608    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpq6l\" (UniqueName: \"kubernetes.io/projected/2f042011-b64a-4346-b37b-ffc5acb0dac2-kube-api-access-zpq6l\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:09:01.384613 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:01.384579    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"
Jul 17 10:09:01.385098 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:01.384614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx" event={"ID":"2f042011-b64a-4346-b37b-ffc5acb0dac2","Type":"ContainerDied","Data":"b3d734c1daba11d7f8a2afef2aeb31aa03f43bf4fa40d60a2c3abec158c8bf82"}
Jul 17 10:09:01.385098 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:01.384673    2575 scope.go:117] "RemoveContainer" containerID="291e6d2b5c39a0461dafb4b93be3dead9b9263fa9c6327721a118a78d4cbab7b"
Jul 17 10:09:01.406209 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:01.406180    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:09:01.407836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:01.407802    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-439eab-88d498947-2rmcx"]
Jul 17 10:09:03.153972 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:03.153938    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f042011-b64a-4346-b37b-ffc5acb0dac2" path="/var/lib/kubelet/pods/2f042011-b64a-4346-b37b-ffc5acb0dac2/volumes"
Jul 17 10:09:05.677482 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677446    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677784    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f042011-b64a-4346-b37b-ffc5acb0dac2" containerName="maas-api"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677797    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f042011-b64a-4346-b37b-ffc5acb0dac2" containerName="maas-api"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677809    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d05e1bf-b1bf-4584-9756-128563dba320" containerName="maas-api"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677815    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d05e1bf-b1bf-4584-9756-128563dba320" containerName="maas-api"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677827    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677833    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677842    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677846    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677854    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.677857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677860    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677871    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677879    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677929    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677937    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677944    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677949    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f042011-b64a-4346-b37b-ffc5acb0dac2" containerName="maas-api"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677955    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677962    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7ab5fa-c05b-40dd-9e5b-0cfa29dbc5c8" containerName="istio-proxy"
Jul 17 10:09:05.678165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.677968    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d05e1bf-b1bf-4584-9756-128563dba320" containerName="maas-api"
Jul 17 10:09:05.682371 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.682351    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:05.685043 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.685022    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:09:05.685236 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.685222    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:09:05.690012 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.689984    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:05.731977 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.731951    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg\" (UID: \"9f1a0e7f-f044-496f-9214-cb8b23906a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:05.832744 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.832706    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg\" (UID: \"9f1a0e7f-f044-496f-9214-cb8b23906a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:05.833326 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.833283    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg\" (UID: \"9f1a0e7f-f044-496f-9214-cb8b23906a8e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:05.994248 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:05.994223    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:06.139190 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:06.139157    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:06.143257 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:09:06.143213    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1a0e7f_f044_496f_9214_cb8b23906a8e.slice/crio-41930b71f5fd3343750b7809619c5146570885a73d02f4061dfcbef3616bb7f2 WatchSource:0}: Error finding container 41930b71f5fd3343750b7809619c5146570885a73d02f4061dfcbef3616bb7f2: Status 404 returned error can't find the container with id 41930b71f5fd3343750b7809619c5146570885a73d02f4061dfcbef3616bb7f2
Jul 17 10:09:06.411020 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:06.410991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerStarted","Data":"41930b71f5fd3343750b7809619c5146570885a73d02f4061dfcbef3616bb7f2"}
Jul 17 10:09:07.417350 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:07.417318    2575 generic.go:356] "Generic (PLEG): container finished" podID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerID="283eaa6c8896823d323da388be77ee51cd214d18821c7480d9928b327d5d5629" exitCode=6
Jul 17 10:09:07.417737 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:07.417391    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerDied","Data":"283eaa6c8896823d323da388be77ee51cd214d18821c7480d9928b327d5d5629"}
Jul 17 10:09:07.417737 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:07.417530    2575 scope.go:117] "RemoveContainer" containerID="283eaa6c8896823d323da388be77ee51cd214d18821c7480d9928b327d5d5629"
Jul 17 10:09:08.423545 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:08.423459    2575 generic.go:356] "Generic (PLEG): container finished" podID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerID="d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2" exitCode=6
Jul 17 10:09:08.423545 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:08.423527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerDied","Data":"d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2"}
Jul 17 10:09:08.424061 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:08.423597    2575 scope.go:117] "RemoveContainer" containerID="283eaa6c8896823d323da388be77ee51cd214d18821c7480d9928b327d5d5629"
Jul 17 10:09:08.424061 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:08.423770    2575 scope.go:117] "RemoveContainer" containerID="d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2"
Jul 17 10:09:08.424061 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:09:08.424010    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg_odh-ai-gateway-infra(9f1a0e7f-f044-496f-9214-cb8b23906a8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e"
Jul 17 10:09:09.430227 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:09.430199    2575 scope.go:117] "RemoveContainer" containerID="d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2"
Jul 17 10:09:09.430604 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:09:09.430407    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg_odh-ai-gateway-infra(9f1a0e7f-f044-496f-9214-cb8b23906a8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e"
Jul 17 10:09:16.084259 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.084219    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:09:16.084742 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.084617    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:16.084742 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.084630    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e9e107-347c-40e1-9632-4792675d5730" containerName="revoke-keys"
Jul 17 10:09:16.088962 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.088943    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.092558 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.092538    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8ae1eb-openshift-default-dockercfg-pnmdc\""
Jul 17 10:09:16.125212 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.125190    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:09:16.226734 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226829    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwphk\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.226926 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.227274 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.226964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.227274 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.227118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327568 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327539    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327592    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327642    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327667    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327686    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwphk\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.327755 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.327754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.328193 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.328088    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.328257 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.328226    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.328355 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.328331    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.328426 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.328406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.328583 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.328566    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.329828 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.329806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.329928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.329850    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.352536 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.352485    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.359978 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.359955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwphk\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk\") pod \"e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") " pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.398991 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.398970    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:16.550414 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.550388    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:09:16.551459 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:09:16.551433    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba83c079_5f5a_4b11_9c1d_f270c15b15a1.slice/crio-cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671 WatchSource:0}: Error finding container cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671: Status 404 returned error can't find the container with id cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671
Jul 17 10:09:16.553572 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.553538    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:16.553681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.553618    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:16.553681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:16.553661    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:17.469834 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:17.469794    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerStarted","Data":"fa78626604319393e67b87f7861bd90763208f0a94974cda8ab18809b9269a05"}
Jul 17 10:09:17.470275 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:17.469876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerStarted","Data":"cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671"}
Jul 17 10:09:17.495734 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:17.495680    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podStartSLOduration=1.495664588 podStartE2EDuration="1.495664588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:17.4940054 +0000 UTC m=+3070.885998018" watchObservedRunningTime="2026-07-17 10:09:17.495664588 +0000 UTC m=+3070.887657204"
Jul 17 10:09:18.399996 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:18.399962    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:18.404707 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:18.404683    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:18.475115 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:18.475087    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:18.475952 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:18.475933    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:19.657258 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.657223    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:09:19.660628 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.660610    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.663732 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.663707    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8ae1eb-serving-cert\""
Jul 17 10:09:19.663857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.663837    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 10:09:19.664829 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.664809    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 10:09:19.686670 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.686647    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:09:19.757949 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.757917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.758150 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.757980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.758150 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.758057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjxn2\" (UniqueName: \"kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.858592 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.858556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.858776 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.858622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.858776 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.858677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjxn2\" (UniqueName: \"kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.859345 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.859326    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.861082 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.861065    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.872224 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.872200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjxn2\" (UniqueName: \"kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2\") pod \"maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:19.971310 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:19.971263    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:20.106076 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:20.106032    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:09:20.108231 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:09:20.108204    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b37110_437c_4688_99c9_3622952ca4a3.slice/crio-5885d619f07c2464a37b68ffb0059c21265fff358e53417d20e08fbc8aa48a1e WatchSource:0}: Error finding container 5885d619f07c2464a37b68ffb0059c21265fff358e53417d20e08fbc8aa48a1e: Status 404 returned error can't find the container with id 5885d619f07c2464a37b68ffb0059c21265fff358e53417d20e08fbc8aa48a1e
Jul 17 10:09:20.147618 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:20.147594    2575 scope.go:117] "RemoveContainer" containerID="d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2"
Jul 17 10:09:20.486760 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:20.486724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerStarted","Data":"d7bbb92778f40ae9309876f73ecd767b7ecee3fa3d9445a6901cd733da59d39e"}
Jul 17 10:09:20.487969 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:20.487943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" event={"ID":"11b37110-437c-4688-99c9-3622952ca4a3","Type":"ContainerStarted","Data":"5885d619f07c2464a37b68ffb0059c21265fff358e53417d20e08fbc8aa48a1e"}
Jul 17 10:09:20.509221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:20.509162    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" podStartSLOduration=15.509149602 podStartE2EDuration="15.509149602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:20.507178497 +0000 UTC m=+3073.899171114" watchObservedRunningTime="2026-07-17 10:09:20.509149602 +0000 UTC m=+3073.901142219"
Jul 17 10:09:21.494872 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.494836    2575 generic.go:356] "Generic (PLEG): container finished" podID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerID="d7bbb92778f40ae9309876f73ecd767b7ecee3fa3d9445a6901cd733da59d39e" exitCode=6
Jul 17 10:09:21.495317 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.494904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerDied","Data":"d7bbb92778f40ae9309876f73ecd767b7ecee3fa3d9445a6901cd733da59d39e"}
Jul 17 10:09:21.495317 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.494960    2575 scope.go:117] "RemoveContainer" containerID="d3c3e78713a8aa8fd4a65979c5081437402fa40780dea723d41d7ec8637bdec2"
Jul 17 10:09:21.495317 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.495224    2575 scope.go:117] "RemoveContainer" containerID="d7bbb92778f40ae9309876f73ecd767b7ecee3fa3d9445a6901cd733da59d39e"
Jul 17 10:09:21.495492 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:09:21.495468    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg_odh-ai-gateway-infra(9f1a0e7f-f044-496f-9214-cb8b23906a8e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e"
Jul 17 10:09:21.496656 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.496631    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" event={"ID":"11b37110-437c-4688-99c9-3622952ca4a3","Type":"ContainerStarted","Data":"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"}
Jul 17 10:09:21.496760 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.496685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:21.534209 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.534181    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:21.591959 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:21.591914    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" podStartSLOduration=2.192064047 podStartE2EDuration="2.591898223s" totalImagesPullingTime="399.834176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:21.588519786 +0000 UTC m=+3074.980512403" watchObservedRunningTime="2026-07-17 10:09:21.591898223 +0000 UTC m=+3074.983890876"
Jul 17 10:09:22.635195 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:22.635170    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:22.787002 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:22.786915    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca\") pod \"9f1a0e7f-f044-496f-9214-cb8b23906a8e\" (UID: \"9f1a0e7f-f044-496f-9214-cb8b23906a8e\") "
Jul 17 10:09:22.787362 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:22.787332    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f1a0e7f-f044-496f-9214-cb8b23906a8e" (UID: "9f1a0e7f-f044-496f-9214-cb8b23906a8e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:22.888207 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:22.888174    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f1a0e7f-f044-496f-9214-cb8b23906a8e-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:09:23.508573 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:23.508536    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg" event={"ID":"9f1a0e7f-f044-496f-9214-cb8b23906a8e","Type":"ContainerDied","Data":"41930b71f5fd3343750b7809619c5146570885a73d02f4061dfcbef3616bb7f2"}
Jul 17 10:09:23.508741 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:23.508581    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"
Jul 17 10:09:23.508741 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:23.508611    2575 scope.go:117] "RemoveContainer" containerID="d7bbb92778f40ae9309876f73ecd767b7ecee3fa3d9445a6901cd733da59d39e"
Jul 17 10:09:23.542917 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:23.542885    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:23.547960 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:23.547937    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-blrvg"]
Jul 17 10:09:25.153367 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:25.153337    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" path="/var/lib/kubelet/pods/9f1a0e7f-f044-496f-9214-cb8b23906a8e/volumes"
Jul 17 10:09:27.508279 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.508252    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:09:27.674810 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.674772    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:09:27.675206 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675188    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675208    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675220    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675229    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675241    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675315 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675249    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675585 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675416    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675585 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675430    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.675585 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.675442    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1a0e7f-f044-496f-9214-cb8b23906a8e" containerName="revoke-keys"
Jul 17 10:09:27.690387 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.690361    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.702371 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.696535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-8d12ab-serving-cert\""
Jul 17 10:09:27.708688 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.708663    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:09:27.836367 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.836259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dph2b\" (UniqueName: \"kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.836367 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.836324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.836568 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.836486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.937366 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.937337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dph2b\" (UniqueName: \"kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.937544 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.937373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.937544 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.937475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.938166 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.938147    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.939829 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.939808    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:27.945883 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:27.945861    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dph2b\" (UniqueName: \"kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b\") pod \"maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:28.012065 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:28.012033    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:28.170433 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:28.170412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:09:28.172663 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:09:28.172634    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9715e979_082e_4a00_899e_76ab49178382.slice/crio-d9382ce80d1d097cd5095ff57e950f6ab0511a8009c8272173e97e9440aaa4d2 WatchSource:0}: Error finding container d9382ce80d1d097cd5095ff57e950f6ab0511a8009c8272173e97e9440aaa4d2: Status 404 returned error can't find the container with id d9382ce80d1d097cd5095ff57e950f6ab0511a8009c8272173e97e9440aaa4d2
Jul 17 10:09:28.540982 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:28.540939    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" event={"ID":"9715e979-082e-4a00-899e-76ab49178382","Type":"ContainerStarted","Data":"d9382ce80d1d097cd5095ff57e950f6ab0511a8009c8272173e97e9440aaa4d2"}
Jul 17 10:09:29.546868 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:29.546834    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" event={"ID":"9715e979-082e-4a00-899e-76ab49178382","Type":"ContainerStarted","Data":"0d1d4cec56feea5c342dc56698c203622ad8460465b3822d904fcf0c7d02ee63"}
Jul 17 10:09:29.547255 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:29.546944    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:29.572209 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:29.572164    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" podStartSLOduration=2.11446575 podStartE2EDuration="2.572149716s" totalImagesPullingTime="457.683966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:29.569760886 +0000 UTC m=+3082.961753506" watchObservedRunningTime="2026-07-17 10:09:29.572149716 +0000 UTC m=+3082.964142332"
Jul 17 10:09:32.562505 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.562435    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerID="fa78626604319393e67b87f7861bd90763208f0a94974cda8ab18809b9269a05" exitCode=0
Jul 17 10:09:32.562885 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.562510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerDied","Data":"fa78626604319393e67b87f7861bd90763208f0a94974cda8ab18809b9269a05"}
Jul 17 10:09:32.562932 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.562888    2575 scope.go:117] "RemoveContainer" containerID="fa78626604319393e67b87f7861bd90763208f0a94974cda8ab18809b9269a05"
Jul 17 10:09:32.563846 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.563818    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:32.563910 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.563886    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:32.563945 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:32.563921    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 17 10:09:33.475849 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:33.475740    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:33.570670 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:33.570632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerStarted","Data":"3cdf3bcd9cda0c7c306d7badcbc38d3f66218026b89a717ab34d65a6c6411327"}
Jul 17 10:09:34.400262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:34.400222    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:34.401098 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:34.401061    2575 patch_prober.go:28] interesting pod/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 10:09:34.401246 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:34.401120    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 10:09:35.400096 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:35.400053    2575 patch_prober.go:28] interesting pod/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 10:09:35.400627 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:35.400387    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 10:09:35.556910 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:35.556882    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:09:36.400162 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:36.400128    2575 patch_prober.go:28] interesting pod/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 10:09:36.400671 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:36.400381    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 10:09:37.399878 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:37.399840    2575 patch_prober.go:28] interesting pod/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Jul 17 10:09:37.400069 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:37.399905    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Jul 17 10:09:38.402986 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:38.402952    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:38.403396 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:38.403222    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:38.403972 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:38.403952    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:09:55.402259 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.402223    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:09:55.405990 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.405968    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.408747 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.408723    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"kube-root-ca.crt\""
Jul 17 10:09:55.408857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.408789    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"auth-test-model-8ae1eb-kserve-self-signed-certs\""
Jul 17 10:09:55.409786 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.409767    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"openshift-service-ca.crt\""
Jul 17 10:09:55.409882 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.409861    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"default-dockercfg-q2mhr\""
Jul 17 10:09:55.415893 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.415872    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:09:55.503066 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rshmz\" (UniqueName: \"kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.503066 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.503235 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.503269 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.503334 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503315    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.503374 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.503339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604703 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604725    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604776    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rshmz\" (UniqueName: \"kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604794    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.604854 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.604812    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.605187 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.605166    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.605262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.605199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.605333 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.605258    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.607034 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.607011    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.607344 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.607323    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.614230 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.614208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rshmz\" (UniqueName: \"kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz\") pod \"auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") " pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.717550 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.717484    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:09:55.852940 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:55.852916    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:09:55.855442 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:09:55.855413    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01d72bb3_4e14_4c5b_9d76_ceb072c60937.slice/crio-3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25 WatchSource:0}: Error finding container 3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25: Status 404 returned error can't find the container with id 3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25
Jul 17 10:09:56.686620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:56.686585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" event={"ID":"01d72bb3-4e14-4c5b-9d76-ceb072c60937","Type":"ContainerStarted","Data":"34fac4ebe449ce4d337181e8fafc4cea4f5c75db2fb797a5966276f81f968e17"}
Jul 17 10:09:56.686620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:56.686631    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" event={"ID":"01d72bb3-4e14-4c5b-9d76-ceb072c60937","Type":"ContainerStarted","Data":"3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25"}
Jul 17 10:09:56.716394 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:09:56.716334    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" podStartSLOduration=1.7163207630000001 podStartE2EDuration="1.716320763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:56.713655986 +0000 UTC m=+3110.105648603" watchObservedRunningTime="2026-07-17 10:09:56.716320763 +0000 UTC m=+3110.108313380"
Jul 17 10:10:05.717696 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:10:05.717665    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:10:05.718208 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:10:05.717864    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:10:05.730731 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:10:05.730703    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:10:06.745160 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:10:06.745130    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:12:50.427921 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:12:50.427890    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:13:05.447709 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.447647    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" podUID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" containerName="main" containerID="cri-o://34fac4ebe449ce4d337181e8fafc4cea4f5c75db2fb797a5966276f81f968e17" gracePeriod=45
Jul 17 10:13:05.598527 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.598491    2575 generic.go:356] "Generic (PLEG): container finished" podID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" containerID="34fac4ebe449ce4d337181e8fafc4cea4f5c75db2fb797a5966276f81f968e17" exitCode=0
Jul 17 10:13:05.598675 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.598575    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" event={"ID":"01d72bb3-4e14-4c5b-9d76-ceb072c60937","Type":"ContainerDied","Data":"34fac4ebe449ce4d337181e8fafc4cea4f5c75db2fb797a5966276f81f968e17"}
Jul 17 10:13:05.701201 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.701140    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:13:05.767514 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767482    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767522    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rshmz\" (UniqueName: \"kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767542    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767616    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767662    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home\") pod \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\" (UID: \"01d72bb3-4e14-4c5b-9d76-ceb072c60937\") "
Jul 17 10:13:05.767939 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767782    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache" (OuterVolumeSpecName: "model-cache") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:13:05.767939 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.767873    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:13:05.768060 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.768000    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home" (OuterVolumeSpecName: "home") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:13:05.768113 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.768069    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-home\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:05.768113 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.768090    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-model-cache\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:05.768183 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.768111    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tmp-dir\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:05.769846 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.769818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm" (OuterVolumeSpecName: "dshm") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:13:05.769846 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.769823    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz" (OuterVolumeSpecName: "kube-api-access-rshmz") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "kube-api-access-rshmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:13:05.769846 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.769830    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01d72bb3-4e14-4c5b-9d76-ceb072c60937" (UID: "01d72bb3-4e14-4c5b-9d76-ceb072c60937"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:13:05.869620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.869592    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rshmz\" (UniqueName: \"kubernetes.io/projected/01d72bb3-4e14-4c5b-9d76-ceb072c60937-kube-api-access-rshmz\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:05.869620 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.869618    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d72bb3-4e14-4c5b-9d76-ceb072c60937-dshm\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:05.869792 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:05.869628    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d72bb3-4e14-4c5b-9d76-ceb072c60937-tls-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:13:06.608853 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:06.608809    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"
Jul 17 10:13:06.608853 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:06.608836    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv" event={"ID":"01d72bb3-4e14-4c5b-9d76-ceb072c60937","Type":"ContainerDied","Data":"3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25"}
Jul 17 10:13:06.609461 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:06.608886    2575 scope.go:117] "RemoveContainer" containerID="34fac4ebe449ce4d337181e8fafc4cea4f5c75db2fb797a5966276f81f968e17"
Jul 17 10:13:06.641931 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:13:06.641901    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01d72bb3_4e14_4c5b_9d76_ceb072c60937.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01d72bb3_4e14_4c5b_9d76_ceb072c60937.slice/crio-3371038a8387eead28e6750defc236d2c71a484eaa2f2f857e80d4476da8eb25\": RecentStats: unable to find data in memory cache]"
Jul 17 10:13:06.650239 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:06.650212    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:13:06.653996 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:06.653973    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/auth-test-model-8ae1eb-kserve-595896ddcf-f6ncv"]
Jul 17 10:13:07.153431 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:07.153398    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" path="/var/lib/kubelet/pods/01d72bb3-4e14-4c5b-9d76-ceb072c60937/volumes"
Jul 17 10:13:07.361134 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:07.361108    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:13:07.366828 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:07.366799    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:13:07.370973 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:07.370957    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:13:07.382975 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:13:07.382955    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lhrpg_0badef41-1489-4263-a9b7-489f7421d5c1/ovn-acl-logging/0.log"
Jul 17 10:14:16.232585 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.232548    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:14:16.233046 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.232985    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" containerName="main"
Jul 17 10:14:16.233046 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.232999    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" containerName="main"
Jul 17 10:14:16.233121 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.233058    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="01d72bb3-4e14-4c5b-9d76-ceb072c60937" containerName="main"
Jul 17 10:14:16.236169 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.236150    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.243837 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.243820    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"default-dockercfg-q2mhr\""
Jul 17 10:14:16.244084 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.244065    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"kube-root-ca.crt\""
Jul 17 10:14:16.244930 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.244912    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"rate-test-model-8ae1eb-kserve-self-signed-certs\""
Jul 17 10:14:16.245724 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.245710    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"openshift-service-ca.crt\""
Jul 17 10:14:16.264525 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.264493    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:14:16.309889 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.309852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.310094 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.309920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.310094 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.310025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.310094 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.310083    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.310281 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.310166    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqvg\" (UniqueName: \"kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.310281 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.310206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411140 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqvg\" (UniqueName: \"kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411140 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411405 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411405 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411405 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411244    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411405 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411267    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411721 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411697    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411795 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.411795 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.411759    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.413554 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.413526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.413800 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.413780    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.422192 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.422165    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqvg\" (UniqueName: \"kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg\") pod \"rate-test-model-8ae1eb-kserve-9975b9955-jq877\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") " pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.546008 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.545917    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:16.691203 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.691180    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:14:16.694038 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:14:16.694011    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11b43da8_6111_48e6_a764_669d0b730c04.slice/crio-5ebde6838a6b2e5453ccb58b3ebb89c8c09b878162c406caaff40dad76736ef3 WatchSource:0}: Error finding container 5ebde6838a6b2e5453ccb58b3ebb89c8c09b878162c406caaff40dad76736ef3: Status 404 returned error can't find the container with id 5ebde6838a6b2e5453ccb58b3ebb89c8c09b878162c406caaff40dad76736ef3
Jul 17 10:14:16.947435 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.947328    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" event={"ID":"11b43da8-6111-48e6-a764-669d0b730c04","Type":"ContainerStarted","Data":"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"}
Jul 17 10:14:16.947435 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.947378    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" event={"ID":"11b43da8-6111-48e6-a764-669d0b730c04","Type":"ContainerStarted","Data":"5ebde6838a6b2e5453ccb58b3ebb89c8c09b878162c406caaff40dad76736ef3"}
Jul 17 10:14:16.972340 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:16.972274    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" podStartSLOduration=0.972259442 podStartE2EDuration="972.259442ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:16.970477199 +0000 UTC m=+3370.362469817" watchObservedRunningTime="2026-07-17 10:14:16.972259442 +0000 UTC m=+3370.364252059"
Jul 17 10:14:26.546612 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:26.546578    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:26.546612 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:26.546616    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:26.558916 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:26.558888    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:14:27.009574 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:14:27.009548    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:15:33.745876 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:33.745844    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:15:36.842957 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:36.842917    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7220-139733724388400"]
Jul 17 10:15:36.846434 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:36.846416    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:36.849490 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:36.849459    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8l869\""
Jul 17 10:15:36.856063 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:36.856029    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724388400"]
Jul 17 10:15:36.953046 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:36.953004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhvb\" (UniqueName: \"kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb\") pod \"test-curl-7220-139733724388400\" (UID: \"18431ff4-5afd-4545-ad3d-a9cc1c0818c4\") " pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:37.054607 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.054572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhvb\" (UniqueName: \"kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb\") pod \"test-curl-7220-139733724388400\" (UID: \"18431ff4-5afd-4545-ad3d-a9cc1c0818c4\") " pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:37.064576 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.064543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhvb\" (UniqueName: \"kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb\") pod \"test-curl-7220-139733724388400\" (UID: \"18431ff4-5afd-4545-ad3d-a9cc1c0818c4\") " pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:37.156976 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.156901    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:37.292370 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.292342    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724388400"]
Jul 17 10:15:37.297569 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.297538    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:15:37.337774 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:37.337750    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724388400" event={"ID":"18431ff4-5afd-4545-ad3d-a9cc1c0818c4","Type":"ContainerStarted","Data":"e7ba5a0949c9e4c95e6678dfc6f145de1b90bb1d7b994dd887b8d2b34cb10e3a"}
Jul 17 10:15:38.343833 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:38.343753    2575 generic.go:356] "Generic (PLEG): container finished" podID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" containerID="850b191161fe9f12fc50b7389bdf3b168ce6d85364cf009dba20b22e314072c8" exitCode=0
Jul 17 10:15:38.343833 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:38.343793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724388400" event={"ID":"18431ff4-5afd-4545-ad3d-a9cc1c0818c4","Type":"ContainerDied","Data":"850b191161fe9f12fc50b7389bdf3b168ce6d85364cf009dba20b22e314072c8"}
Jul 17 10:15:39.482395 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.482366    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:39.521024 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.520993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7220-139733724388400_18431ff4-5afd-4545-ad3d-a9cc1c0818c4/test-curl-7220-139733724388400/0.log"
Jul 17 10:15:39.551225 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.551198    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7220-139733724388400"]
Jul 17 10:15:39.555111 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.555089    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7220-139733724388400"]
Jul 17 10:15:39.578242 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.578217    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdhvb\" (UniqueName: \"kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb\") pod \"18431ff4-5afd-4545-ad3d-a9cc1c0818c4\" (UID: \"18431ff4-5afd-4545-ad3d-a9cc1c0818c4\") "
Jul 17 10:15:39.580355 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.580330    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb" (OuterVolumeSpecName: "kube-api-access-mdhvb") pod "18431ff4-5afd-4545-ad3d-a9cc1c0818c4" (UID: "18431ff4-5afd-4545-ad3d-a9cc1c0818c4"). InnerVolumeSpecName "kube-api-access-mdhvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:15:39.679677 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:39.679605    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdhvb\" (UniqueName: \"kubernetes.io/projected/18431ff4-5afd-4545-ad3d-a9cc1c0818c4-kube-api-access-mdhvb\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:40.354606 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:40.354574    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:40.354804 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:40.354576    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7ba5a0949c9e4c95e6678dfc6f145de1b90bb1d7b994dd887b8d2b34cb10e3a"
Jul 17 10:15:40.365353 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:15:40.365323    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7220-139733724388400\" is forbidden: User \"system:node:ip-10-0-139-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-140.ec2.internal' and this object" podUID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" pod="opendatahub/test-curl-7220-139733724388400"
Jul 17 10:15:41.153241 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:41.153210    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" path="/var/lib/kubelet/pods/18431ff4-5afd-4545-ad3d-a9cc1c0818c4/volumes"
Jul 17 10:15:42.642785 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.642754    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:42.643158 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.643115    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" containerName="test-curl-7220-139733724388400"
Jul 17 10:15:42.643158 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.643127    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" containerName="test-curl-7220-139733724388400"
Jul 17 10:15:42.643235 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.643195    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="18431ff4-5afd-4545-ad3d-a9cc1c0818c4" containerName="test-curl-7220-139733724388400"
Jul 17 10:15:42.646393 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.646375    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:42.649658 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.649639    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8l869\""
Jul 17 10:15:42.657090 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.657064    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:42.807961 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.807925    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zllxr\" (UniqueName: \"kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr\") pod \"test-curl-7220-139733724415584\" (UID: \"adc4817e-0ad1-417f-921e-e3a38fef6835\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:42.909307 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.909215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zllxr\" (UniqueName: \"kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr\") pod \"test-curl-7220-139733724415584\" (UID: \"adc4817e-0ad1-417f-921e-e3a38fef6835\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:42.920130 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.920101    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zllxr\" (UniqueName: \"kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr\") pod \"test-curl-7220-139733724415584\" (UID: \"adc4817e-0ad1-417f-921e-e3a38fef6835\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:42.956450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:42.956410    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:43.093878 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:43.093855    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:43.096203 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:15:43.096176    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadc4817e_0ad1_417f_921e_e3a38fef6835.slice/crio-44e555a873ab27648671e8d63e7a07271b4c6f724f1358ae99577a2d533c78c0 WatchSource:0}: Error finding container 44e555a873ab27648671e8d63e7a07271b4c6f724f1358ae99577a2d533c78c0: Status 404 returned error can't find the container with id 44e555a873ab27648671e8d63e7a07271b4c6f724f1358ae99577a2d533c78c0
Jul 17 10:15:43.369348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:43.369288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724415584" event={"ID":"adc4817e-0ad1-417f-921e-e3a38fef6835","Type":"ContainerStarted","Data":"44e555a873ab27648671e8d63e7a07271b4c6f724f1358ae99577a2d533c78c0"}
Jul 17 10:15:44.376009 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:44.375971    2575 generic.go:356] "Generic (PLEG): container finished" podID="adc4817e-0ad1-417f-921e-e3a38fef6835" containerID="898a3bca3d7ecc197acb9c546f9df5b5449391cf1fa4500a46a6cdad396fb97c" exitCode=0
Jul 17 10:15:44.376408 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:44.376067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724415584" event={"ID":"adc4817e-0ad1-417f-921e-e3a38fef6835","Type":"ContainerDied","Data":"898a3bca3d7ecc197acb9c546f9df5b5449391cf1fa4500a46a6cdad396fb97c"}
Jul 17 10:15:45.512450 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.512427    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:45.533080 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.533004    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7220-139733724415584_adc4817e-0ad1-417f-921e-e3a38fef6835/test-curl-7220-139733724415584/0.log"
Jul 17 10:15:45.558973 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.558944    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:45.562934 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.562911    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:45.634124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.634063    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zllxr\" (UniqueName: \"kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr\") pod \"adc4817e-0ad1-417f-921e-e3a38fef6835\" (UID: \"adc4817e-0ad1-417f-921e-e3a38fef6835\") "
Jul 17 10:15:45.636113 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.636084    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr" (OuterVolumeSpecName: "kube-api-access-zllxr") pod "adc4817e-0ad1-417f-921e-e3a38fef6835" (UID: "adc4817e-0ad1-417f-921e-e3a38fef6835"). InnerVolumeSpecName "kube-api-access-zllxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:15:45.680860 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.680836    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:45.681202 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.681190    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc4817e-0ad1-417f-921e-e3a38fef6835" containerName="test-curl-7220-139733724415584"
Jul 17 10:15:45.681256 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.681205    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc4817e-0ad1-417f-921e-e3a38fef6835" containerName="test-curl-7220-139733724415584"
Jul 17 10:15:45.681314 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.681278    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc4817e-0ad1-417f-921e-e3a38fef6835" containerName="test-curl-7220-139733724415584"
Jul 17 10:15:45.684260 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.684241    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:45.691695 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.691673    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:45.735663 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.735637    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zllxr\" (UniqueName: \"kubernetes.io/projected/adc4817e-0ad1-417f-921e-e3a38fef6835-kube-api-access-zllxr\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:45.836719 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.836692    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfz9q\" (UniqueName: \"kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q\") pod \"test-curl-7220-139733724415584\" (UID: \"95cdf90b-e1ed-4faa-890b-0a613c536110\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:45.937778 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.937712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfz9q\" (UniqueName: \"kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q\") pod \"test-curl-7220-139733724415584\" (UID: \"95cdf90b-e1ed-4faa-890b-0a613c536110\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:45.947393 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.947365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfz9q\" (UniqueName: \"kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q\") pod \"test-curl-7220-139733724415584\" (UID: \"95cdf90b-e1ed-4faa-890b-0a613c536110\") " pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:45.994323 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:45.994289    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:46.134800 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:46.134773    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:46.136175 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:15:46.136145    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95cdf90b_e1ed_4faa_890b_0a613c536110.slice/crio-1271b4dea15baa835186c3b44a86d1e3291ac0ed0e9581c7fb7bc3083b50f550 WatchSource:0}: Error finding container 1271b4dea15baa835186c3b44a86d1e3291ac0ed0e9581c7fb7bc3083b50f550: Status 404 returned error can't find the container with id 1271b4dea15baa835186c3b44a86d1e3291ac0ed0e9581c7fb7bc3083b50f550
Jul 17 10:15:46.386749 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:46.386719    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724415584" event={"ID":"95cdf90b-e1ed-4faa-890b-0a613c536110","Type":"ContainerStarted","Data":"1271b4dea15baa835186c3b44a86d1e3291ac0ed0e9581c7fb7bc3083b50f550"}
Jul 17 10:15:46.387928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:46.387907    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:46.388032 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:46.387912    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44e555a873ab27648671e8d63e7a07271b4c6f724f1358ae99577a2d533c78c0"
Jul 17 10:15:46.402361 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:46.402327    2575 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7220-139733724415584" oldPodUID="adc4817e-0ad1-417f-921e-e3a38fef6835" podUID="95cdf90b-e1ed-4faa-890b-0a613c536110"
Jul 17 10:15:47.153469 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:47.153438    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc4817e-0ad1-417f-921e-e3a38fef6835" path="/var/lib/kubelet/pods/adc4817e-0ad1-417f-921e-e3a38fef6835/volumes"
Jul 17 10:15:47.394582 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:47.394548    2575 generic.go:356] "Generic (PLEG): container finished" podID="95cdf90b-e1ed-4faa-890b-0a613c536110" containerID="4bf5d5d8022e4451691a131fbd3e35f38c0a259da604af5ebae7b647f4fc2fea" exitCode=0
Jul 17 10:15:47.394754 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:47.394635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724415584" event={"ID":"95cdf90b-e1ed-4faa-890b-0a613c536110","Type":"ContainerDied","Data":"4bf5d5d8022e4451691a131fbd3e35f38c0a259da604af5ebae7b647f4fc2fea"}
Jul 17 10:15:48.537611 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.537586    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:48.561678 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.561651    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfz9q\" (UniqueName: \"kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q\") pod \"95cdf90b-e1ed-4faa-890b-0a613c536110\" (UID: \"95cdf90b-e1ed-4faa-890b-0a613c536110\") "
Jul 17 10:15:48.563861 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.563831    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q" (OuterVolumeSpecName: "kube-api-access-sfz9q") pod "95cdf90b-e1ed-4faa-890b-0a613c536110" (UID: "95cdf90b-e1ed-4faa-890b-0a613c536110"). InnerVolumeSpecName "kube-api-access-sfz9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:15:48.565595 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.565573    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7220-139733724415584_95cdf90b-e1ed-4faa-890b-0a613c536110/test-curl-7220-139733724415584/0.log"
Jul 17 10:15:48.593864 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.593840    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:48.599093 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.599064    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7220-139733724415584"]
Jul 17 10:15:48.662683 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.662617    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfz9q\" (UniqueName: \"kubernetes.io/projected/95cdf90b-e1ed-4faa-890b-0a613c536110-kube-api-access-sfz9q\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:48.765388 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:48.765347    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" podUID="11b43da8-6111-48e6-a764-669d0b730c04" containerName="main" containerID="cri-o://f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461" gracePeriod=45
Jul 17 10:15:49.021857 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.021832    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:15:49.065986 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.065957    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066129 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.065994    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066129 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066039    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066129 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066058    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066129 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066082    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqqvg\" (UniqueName: \"kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066380 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066136    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home\") pod \"11b43da8-6111-48e6-a764-669d0b730c04\" (UID: \"11b43da8-6111-48e6-a764-669d0b730c04\") "
Jul 17 10:15:49.066380 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066269    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:15:49.066380 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066278    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache" (OuterVolumeSpecName: "model-cache") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:15:49.066564 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066443    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-model-cache\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.066564 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066486    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-tmp-dir\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.066564 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.066505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home" (OuterVolumeSpecName: "home") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:15:49.068227 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.068203    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:15:49.068598 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.068576    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg" (OuterVolumeSpecName: "kube-api-access-kqqvg") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "kube-api-access-kqqvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:15:49.068681 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.068586    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm" (OuterVolumeSpecName: "dshm") pod "11b43da8-6111-48e6-a764-669d0b730c04" (UID: "11b43da8-6111-48e6-a764-669d0b730c04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:15:49.153027 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.152995    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95cdf90b-e1ed-4faa-890b-0a613c536110" path="/var/lib/kubelet/pods/95cdf90b-e1ed-4faa-890b-0a613c536110/volumes"
Jul 17 10:15:49.167682 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.167661    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11b43da8-6111-48e6-a764-669d0b730c04-tls-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.167768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.167683    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqqvg\" (UniqueName: \"kubernetes.io/projected/11b43da8-6111-48e6-a764-669d0b730c04-kube-api-access-kqqvg\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.167768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.167699    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-home\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.167768 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.167711    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11b43da8-6111-48e6-a764-669d0b730c04-dshm\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:15:49.411337 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.411290    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724415584"
Jul 17 10:15:49.411337 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.411340    2575 scope.go:117] "RemoveContainer" containerID="4bf5d5d8022e4451691a131fbd3e35f38c0a259da604af5ebae7b647f4fc2fea"
Jul 17 10:15:49.415836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.414829    2575 generic.go:356] "Generic (PLEG): container finished" podID="11b43da8-6111-48e6-a764-669d0b730c04" containerID="f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461" exitCode=0
Jul 17 10:15:49.415836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.414986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" event={"ID":"11b43da8-6111-48e6-a764-669d0b730c04","Type":"ContainerDied","Data":"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"}
Jul 17 10:15:49.415836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.415020    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877" event={"ID":"11b43da8-6111-48e6-a764-669d0b730c04","Type":"ContainerDied","Data":"5ebde6838a6b2e5453ccb58b3ebb89c8c09b878162c406caaff40dad76736ef3"}
Jul 17 10:15:49.415836 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.415071    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"
Jul 17 10:15:49.422942 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.422922    2575 scope.go:117] "RemoveContainer" containerID="f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"
Jul 17 10:15:49.430595 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.430580    2575 scope.go:117] "RemoveContainer" containerID="f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"
Jul 17 10:15:49.430840 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:15:49.430822    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461\": container with ID starting with f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461 not found: ID does not exist" containerID="f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"
Jul 17 10:15:49.430908 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.430847    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461"} err="failed to get container status \"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461\": rpc error: code = NotFound desc = could not find container \"f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461\": container with ID starting with f128373d363fa6d3e043ea034a12bdf6cca83e37f36aa9411fb35cf18ca4d461 not found: ID does not exist"
Jul 17 10:15:49.435025 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.435004    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:15:49.439562 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:49.439542    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/rate-test-model-8ae1eb-kserve-9975b9955-jq877"]
Jul 17 10:15:51.153042 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:15:51.153008    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b43da8-6111-48e6-a764-669d0b730c04" path="/var/lib/kubelet/pods/11b43da8-6111-48e6-a764-669d0b730c04/volumes"
Jul 17 10:16:05.070252 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.070215    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:16:05.070775 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.070452    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" podUID="11b37110-437c-4688-99c9-3622952ca4a3" containerName="maas-api" containerID="cri-o://305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986" gracePeriod=30
Jul 17 10:16:05.313589 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.313565    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:16:05.414552 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.414474    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjxn2\" (UniqueName: \"kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2\") pod \"11b37110-437c-4688-99c9-3622952ca4a3\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") "
Jul 17 10:16:05.414552 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.414523    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls\") pod \"11b37110-437c-4688-99c9-3622952ca4a3\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") "
Jul 17 10:16:05.414770 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.414566    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca\") pod \"11b37110-437c-4688-99c9-3622952ca4a3\" (UID: \"11b37110-437c-4688-99c9-3622952ca4a3\") "
Jul 17 10:16:05.414978 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.414956    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11b37110-437c-4688-99c9-3622952ca4a3" (UID: "11b37110-437c-4688-99c9-3622952ca4a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:05.416622 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.416601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2" (OuterVolumeSpecName: "kube-api-access-hjxn2") pod "11b37110-437c-4688-99c9-3622952ca4a3" (UID: "11b37110-437c-4688-99c9-3622952ca4a3"). InnerVolumeSpecName "kube-api-access-hjxn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:05.416695 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.416640    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11b37110-437c-4688-99c9-3622952ca4a3" (UID: "11b37110-437c-4688-99c9-3622952ca4a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:16:05.502235 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.502208    2575 generic.go:356] "Generic (PLEG): container finished" podID="11b37110-437c-4688-99c9-3622952ca4a3" containerID="305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986" exitCode=0
Jul 17 10:16:05.502371 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.502247    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"
Jul 17 10:16:05.502371 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.502290    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" event={"ID":"11b37110-437c-4688-99c9-3622952ca4a3","Type":"ContainerDied","Data":"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"}
Jul 17 10:16:05.502371 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.502356    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x" event={"ID":"11b37110-437c-4688-99c9-3622952ca4a3","Type":"ContainerDied","Data":"5885d619f07c2464a37b68ffb0059c21265fff358e53417d20e08fbc8aa48a1e"}
Jul 17 10:16:05.502590 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.502379    2575 scope.go:117] "RemoveContainer" containerID="305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"
Jul 17 10:16:05.511280 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.511262    2575 scope.go:117] "RemoveContainer" containerID="305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"
Jul 17 10:16:05.511535 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:16:05.511517    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986\": container with ID starting with 305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986 not found: ID does not exist" containerID="305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"
Jul 17 10:16:05.511587 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.511542    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986"} err="failed to get container status \"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986\": rpc error: code = NotFound desc = could not find container \"305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986\": container with ID starting with 305271e5b831985a921d946f2bf1ab7a8992f014db7c2e6a973c99772e860986 not found: ID does not exist"
Jul 17 10:16:05.515205 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.515187    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjxn2\" (UniqueName: \"kubernetes.io/projected/11b37110-437c-4688-99c9-3622952ca4a3-kube-api-access-hjxn2\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:05.515287 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.515206    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11b37110-437c-4688-99c9-3622952ca4a3-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:05.515287 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.515217    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b37110-437c-4688-99c9-3622952ca4a3-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:05.525008 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.524986    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:16:05.528947 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:05.528926    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8ae1eb-74765445c5-crn9x"]
Jul 17 10:16:07.153124 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:07.153088    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b37110-437c-4688-99c9-3622952ca4a3" path="/var/lib/kubelet/pods/11b37110-437c-4688-99c9-3622952ca4a3/volumes"
Jul 17 10:16:20.377692 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:20.377660    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:16:20.378355 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:20.377913    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" containerID="cri-o://3cdf3bcd9cda0c7c306d7badcbc38d3f66218026b89a717ab34d65a6c6411327" gracePeriod=30
Jul 17 10:16:23.405943 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:23.405903    2575 patch_prober.go:28] interesting pod/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 10:16:23.406353 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:23.405957    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 10:16:25.607861 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.607833    2575 generic.go:356] "Generic (PLEG): container finished" podID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerID="3cdf3bcd9cda0c7c306d7badcbc38d3f66218026b89a717ab34d65a6c6411327" exitCode=0
Jul 17 10:16:25.608221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.607904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerDied","Data":"3cdf3bcd9cda0c7c306d7badcbc38d3f66218026b89a717ab34d65a6c6411327"}
Jul 17 10:16:25.608221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.607967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v" event={"ID":"ba83c079-5f5a-4b11-9c1d-f270c15b15a1","Type":"ContainerDied","Data":"cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671"}
Jul 17 10:16:25.608221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.607986    2575 scope.go:117] "RemoveContainer" containerID="fa78626604319393e67b87f7861bd90763208f0a94974cda8ab18809b9269a05"
Jul 17 10:16:25.608221 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.607987    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf19cc1d7cd26c10e19c6c05526c1cb394e9e5d5c98fab9bc71b6cf943263671"
Jul 17 10:16:25.621547 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.621529    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:16:25.797780 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797753    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.797946 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797794    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.797946 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797823    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwphk\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.797946 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797903    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798118 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797968    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798118 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.797992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798118 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798035    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798118 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798065    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798118 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798107    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs\") pod \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\" (UID: \"ba83c079-5f5a-4b11-9c1d-f270c15b15a1\") "
Jul 17 10:16:25.798404 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798162    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:25.798404 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798199    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:25.798522 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798462    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:25.798522 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798493    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-credential-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.798522 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798498    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:25.798522 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798515    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istiod-ca-cert\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.799006 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.798969    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:25.800141 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.800118    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk" (OuterVolumeSpecName: "kube-api-access-gwphk") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "kube-api-access-gwphk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:25.800241 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.800140    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:25.800241 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.800165    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:25.800448 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.800422    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba83c079-5f5a-4b11-9c1d-f270c15b15a1" (UID: "ba83c079-5f5a-4b11-9c1d-f270c15b15a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:16:25.899774 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899744    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-podinfo\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899774 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899768    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-token\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899782    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-data\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899794    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-socket\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899808    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-workload-certs\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899819    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-istio-envoy\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.899928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:25.899831    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwphk\" (UniqueName: \"kubernetes.io/projected/ba83c079-5f5a-4b11-9c1d-f270c15b15a1-kube-api-access-gwphk\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:26.613825 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:26.613793    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"
Jul 17 10:16:26.659003 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:26.658968    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:16:26.665921 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:26.665894    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8ae1eb-openshift-default-5bb5c99c4c-s786v"]
Jul 17 10:16:27.153533 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:27.153503    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" path="/var/lib/kubelet/pods/ba83c079-5f5a-4b11-9c1d-f270c15b15a1/volumes"
Jul 17 10:16:31.427079 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.427004    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:16:31.427466 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.427191    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" podUID="9715e979-082e-4a00-899e-76ab49178382" containerName="maas-api" containerID="cri-o://0d1d4cec56feea5c342dc56698c203622ad8460465b3822d904fcf0c7d02ee63" gracePeriod=30
Jul 17 10:16:31.641204 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.641169    2575 generic.go:356] "Generic (PLEG): container finished" podID="9715e979-082e-4a00-899e-76ab49178382" containerID="0d1d4cec56feea5c342dc56698c203622ad8460465b3822d904fcf0c7d02ee63" exitCode=0
Jul 17 10:16:31.641428 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.641243    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" event={"ID":"9715e979-082e-4a00-899e-76ab49178382","Type":"ContainerDied","Data":"0d1d4cec56feea5c342dc56698c203622ad8460465b3822d904fcf0c7d02ee63"}
Jul 17 10:16:31.671552 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.671527    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:16:31.853671 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.853636    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca\") pod \"9715e979-082e-4a00-899e-76ab49178382\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") "
Jul 17 10:16:31.853851 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.853741    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dph2b\" (UniqueName: \"kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b\") pod \"9715e979-082e-4a00-899e-76ab49178382\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") "
Jul 17 10:16:31.853851 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.853790    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls\") pod \"9715e979-082e-4a00-899e-76ab49178382\" (UID: \"9715e979-082e-4a00-899e-76ab49178382\") "
Jul 17 10:16:31.853998 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.853976    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9715e979-082e-4a00-899e-76ab49178382" (UID: "9715e979-082e-4a00-899e-76ab49178382"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:31.855975 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.855955    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9715e979-082e-4a00-899e-76ab49178382" (UID: "9715e979-082e-4a00-899e-76ab49178382"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:16:31.856079 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.856059    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b" (OuterVolumeSpecName: "kube-api-access-dph2b") pod "9715e979-082e-4a00-899e-76ab49178382" (UID: "9715e979-082e-4a00-899e-76ab49178382"). InnerVolumeSpecName "kube-api-access-dph2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:31.955012 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.954979    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9715e979-082e-4a00-899e-76ab49178382-openshift-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:31.955012 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.955007    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dph2b\" (UniqueName: \"kubernetes.io/projected/9715e979-082e-4a00-899e-76ab49178382-kube-api-access-dph2b\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:31.955012 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:31.955020    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9715e979-082e-4a00-899e-76ab49178382-maas-api-tls\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:16:32.647669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:32.647637    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8" event={"ID":"9715e979-082e-4a00-899e-76ab49178382","Type":"ContainerDied","Data":"d9382ce80d1d097cd5095ff57e950f6ab0511a8009c8272173e97e9440aaa4d2"}
Jul 17 10:16:32.647669 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:32.647655    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"
Jul 17 10:16:32.648169 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:32.647696    2575 scope.go:117] "RemoveContainer" containerID="0d1d4cec56feea5c342dc56698c203622ad8460465b3822d904fcf0c7d02ee63"
Jul 17 10:16:32.670655 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:32.670633    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:16:32.674987 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:32.674965    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-8d12ab-5888bb445d-6qzx8"]
Jul 17 10:16:33.153035 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:33.152998    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9715e979-082e-4a00-899e-76ab49178382" path="/var/lib/kubelet/pods/9715e979-082e-4a00-899e-76ab49178382/volumes"
Jul 17 10:16:46.532262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532225    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:16:46.532763 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532746    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9715e979-082e-4a00-899e-76ab49178382" containerName="maas-api"
Jul 17 10:16:46.532805 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532767    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9715e979-082e-4a00-899e-76ab49178382" containerName="maas-api"
Jul 17 10:16:46.532805 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532785    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b43da8-6111-48e6-a764-669d0b730c04" containerName="main"
Jul 17 10:16:46.532805 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532794    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b43da8-6111-48e6-a764-669d0b730c04" containerName="main"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532810    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95cdf90b-e1ed-4faa-890b-0a613c536110" containerName="test-curl-7220-139733724415584"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532821    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="95cdf90b-e1ed-4faa-890b-0a613c536110" containerName="test-curl-7220-139733724415584"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532849    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532857    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532871    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b37110-437c-4688-99c9-3622952ca4a3" containerName="maas-api"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532880    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b37110-437c-4688-99c9-3622952ca4a3" containerName="maas-api"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532889    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.532899 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532898    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532984    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9715e979-082e-4a00-899e-76ab49178382" containerName="maas-api"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.532997    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.533007    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba83c079-5f5a-4b11-9c1d-f270c15b15a1" containerName="istio-proxy"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.533017    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="95cdf90b-e1ed-4faa-890b-0a613c536110" containerName="test-curl-7220-139733724415584"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.533026    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b43da8-6111-48e6-a764-669d0b730c04" containerName="main"
Jul 17 10:16:46.533176 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.533036    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b37110-437c-4688-99c9-3622952ca4a3" containerName="maas-api"
Jul 17 10:16:46.537643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.537623    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:16:46.540684 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.540659    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:16:46.540810 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.540698    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:16:46.545942 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.545845    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:16:46.583653 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.583624    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk\" (UID: \"5491ef40-7d91-441d-8742-29ee905f5919\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:16:46.684643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.684600    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk\" (UID: \"5491ef40-7d91-441d-8742-29ee905f5919\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:16:46.685203 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.685182    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk\" (UID: \"5491ef40-7d91-441d-8742-29ee905f5919\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:16:46.850166 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.850094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:16:46.984069 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:46.984044    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:16:46.986435 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:16:46.986407    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5491ef40_7d91_441d_8742_29ee905f5919.slice/crio-6d34c12faa6173c6d6b7b91acef1a0095b00ac69b2bbd2275902a481350e8a43 WatchSource:0}: Error finding container 6d34c12faa6173c6d6b7b91acef1a0095b00ac69b2bbd2275902a481350e8a43: Status 404 returned error can't find the container with id 6d34c12faa6173c6d6b7b91acef1a0095b00ac69b2bbd2275902a481350e8a43
Jul 17 10:16:47.723824 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:47.723790    2575 generic.go:356] "Generic (PLEG): container finished" podID="5491ef40-7d91-441d-8742-29ee905f5919" containerID="7141da11bf0ea6c9ec79e6c85e9157bd2d83ea887779a806617009373f096494" exitCode=6
Jul 17 10:16:47.724262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:47.723876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerDied","Data":"7141da11bf0ea6c9ec79e6c85e9157bd2d83ea887779a806617009373f096494"}
Jul 17 10:16:47.724262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:47.723933    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerStarted","Data":"6d34c12faa6173c6d6b7b91acef1a0095b00ac69b2bbd2275902a481350e8a43"}
Jul 17 10:16:47.724262 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:47.724066    2575 scope.go:117] "RemoveContainer" containerID="7141da11bf0ea6c9ec79e6c85e9157bd2d83ea887779a806617009373f096494"
Jul 17 10:16:48.730488 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:48.730456    2575 generic.go:356] "Generic (PLEG): container finished" podID="5491ef40-7d91-441d-8742-29ee905f5919" containerID="d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646" exitCode=6
Jul 17 10:16:48.730883 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:48.730548    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerDied","Data":"d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646"}
Jul 17 10:16:48.730883 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:48.730609    2575 scope.go:117] "RemoveContainer" containerID="7141da11bf0ea6c9ec79e6c85e9157bd2d83ea887779a806617009373f096494"
Jul 17 10:16:48.730883 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:48.730678    2575 scope.go:117] "RemoveContainer" containerID="d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646"
Jul 17 10:16:48.731004 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:16:48.730906    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk_odh-ai-gateway-infra(5491ef40-7d91-441d-8742-29ee905f5919)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" podUID="5491ef40-7d91-441d-8742-29ee905f5919"
Jul 17 10:16:49.737369 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:16:49.737345    2575 scope.go:117] "RemoveContainer" containerID="d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646"
Jul 17 10:16:49.737785 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:16:49.737534    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk_odh-ai-gateway-infra(5491ef40-7d91-441d-8742-29ee905f5919)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" podUID="5491ef40-7d91-441d-8742-29ee905f5919"
Jul 17 10:17:04.833579 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:04.833550    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-rhjtm_718156a7-f6a4-4e41-8b63-56c317205082/copy-manifests/0.log"
Jul 17 10:17:04.840782 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:04.840761    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-rhjtm_718156a7-f6a4-4e41-8b63-56c317205082/manager/0.log"
Jul 17 10:17:05.147573 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.147498    2575 scope.go:117] "RemoveContainer" containerID="d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646"
Jul 17 10:17:05.443018 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.442946    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738025-rlc4m_9db63d64-322a-477e-b0e5-6e8ceef76e33/cleanup/0.log"
Jul 17 10:17:05.551817 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.551786    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738040-dvrk8_7d0c9901-4908-451d-b874-09d85393c188/cleanup/0.log"
Jul 17 10:17:05.762574 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.762550    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk_5491ef40-7d91-441d-8742-29ee905f5919/revoke-keys/1.log"
Jul 17 10:17:05.816605 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.816578    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerStarted","Data":"bf8c700684c6dbd7f888b373db3d2da320143188a2f9b29794f1b9f3f1c569bc"}
Jul 17 10:17:05.838030 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:05.837982    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" podStartSLOduration=19.837963753 podStartE2EDuration="19.837963753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:05.837362661 +0000 UTC m=+3539.229355279" watchObservedRunningTime="2026-07-17 10:17:05.837963753 +0000 UTC m=+3539.229956383"
Jul 17 10:17:06.823263 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:06.823232    2575 generic.go:356] "Generic (PLEG): container finished" podID="5491ef40-7d91-441d-8742-29ee905f5919" containerID="bf8c700684c6dbd7f888b373db3d2da320143188a2f9b29794f1b9f3f1c569bc" exitCode=6
Jul 17 10:17:06.823462 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:06.823322    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerDied","Data":"bf8c700684c6dbd7f888b373db3d2da320143188a2f9b29794f1b9f3f1c569bc"}
Jul 17 10:17:06.823462 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:06.823384    2575 scope.go:117] "RemoveContainer" containerID="d86fa805e6d62f06c387ca6c577f1da682daebc1d99fc2a4c38ce8292f8de646"
Jul 17 10:17:06.823657 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:06.823632    2575 scope.go:117] "RemoveContainer" containerID="bf8c700684c6dbd7f888b373db3d2da320143188a2f9b29794f1b9f3f1c569bc"
Jul 17 10:17:06.823844 ip-10-0-139-140 kubenswrapper[2575]: E0717 10:17:06.823824    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk_odh-ai-gateway-infra(5491ef40-7d91-441d-8742-29ee905f5919)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" podUID="5491ef40-7d91-441d-8742-29ee905f5919"
Jul 17 10:17:06.851928 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:06.851898    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:17:07.550432 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:07.550402    2575 scope.go:117] "RemoveContainer" containerID="3cdf3bcd9cda0c7c306d7badcbc38d3f66218026b89a717ab34d65a6c6411327"
Jul 17 10:17:07.960379 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:07.960352    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:17:08.068846 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.068818    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca\") pod \"5491ef40-7d91-441d-8742-29ee905f5919\" (UID: \"5491ef40-7d91-441d-8742-29ee905f5919\") "
Jul 17 10:17:08.069245 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.069223    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5491ef40-7d91-441d-8742-29ee905f5919" (UID: "5491ef40-7d91-441d-8742-29ee905f5919"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:08.170448 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.170372    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5491ef40-7d91-441d-8742-29ee905f5919-maas-api-service-ca\") on node \"ip-10-0-139-140.ec2.internal\" DevicePath \"\""
Jul 17 10:17:08.440563 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.440479    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c4ntr_3de5614f-4875-48eb-b5aa-381bd6d167af/kuadrant-console-plugin/0.log"
Jul 17 10:17:08.835068 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.835041    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"
Jul 17 10:17:08.835243 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.835090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk" event={"ID":"5491ef40-7d91-441d-8742-29ee905f5919","Type":"ContainerDied","Data":"6d34c12faa6173c6d6b7b91acef1a0095b00ac69b2bbd2275902a481350e8a43"}
Jul 17 10:17:08.835243 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.835138    2575 scope.go:117] "RemoveContainer" containerID="bf8c700684c6dbd7f888b373db3d2da320143188a2f9b29794f1b9f3f1c569bc"
Jul 17 10:17:08.857386 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.857357    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:17:08.862426 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:08.862406    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-c9zrk"]
Jul 17 10:17:09.152898 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:09.152817    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5491ef40-7d91-441d-8742-29ee905f5919" path="/var/lib/kubelet/pods/5491ef40-7d91-441d-8742-29ee905f5919/volumes"
Jul 17 10:17:09.239046 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:09.239020    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45pw68_ce89b7b3-a842-4f44-894f-203708d84729/istio-proxy/0.log"
Jul 17 10:17:09.571288 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:09.571257    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-qznjf_8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a/kube-auth-proxy/0.log"
Jul 17 10:17:10.598705 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:10.598674    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ls46t_9f1cf106-17a4-4882-9157-e66acf54a339/main/0.log"
Jul 17 10:17:10.712532 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:10.712506    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vmbk_f5d0c117-28f7-4646-8131-2784d20132fa/main/0.log"
Jul 17 10:17:10.938193 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:10.938094    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-w62tp_d3dc8aaf-03a1-4751-8290-8d336ceffd41/main/0.log"
Jul 17 10:17:17.842504 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842464    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z5nh8/must-gather-7h7gh"]
Jul 17 10:17:17.842887 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842830    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.842887 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842845    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.842887 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842866    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.842887 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842872    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.843023 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842936    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.843023 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.842946    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.843023 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.843006    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.843023 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.843013    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.843143 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.843085    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5491ef40-7d91-441d-8742-29ee905f5919" containerName="revoke-keys"
Jul 17 10:17:17.847285 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.847264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:17.850053 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.850028    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5nh8\"/\"kube-root-ca.crt\""
Jul 17 10:17:17.850184 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.850157    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5nh8\"/\"openshift-service-ca.crt\""
Jul 17 10:17:17.855685 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.855659    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/must-gather-7h7gh"]
Jul 17 10:17:17.956707 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.956668    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8pst\" (UniqueName: \"kubernetes.io/projected/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-kube-api-access-x8pst\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:17.956901 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:17.956729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-must-gather-output\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.057250 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.057215    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8pst\" (UniqueName: \"kubernetes.io/projected/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-kube-api-access-x8pst\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.057440 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.057264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-must-gather-output\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.057643 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.057621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-must-gather-output\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.067037 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.067015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8pst\" (UniqueName: \"kubernetes.io/projected/1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba-kube-api-access-x8pst\") pod \"must-gather-7h7gh\" (UID: \"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba\") " pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.156345 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.156254    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/must-gather-7h7gh"
Jul 17 10:17:18.298165 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.298134    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/must-gather-7h7gh"]
Jul 17 10:17:18.301410 ip-10-0-139-140 kubenswrapper[2575]: W0717 10:17:18.301380    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a3c3bfe_4f83_4a63_ae8f_d3434bea49ba.slice/crio-bcda9c4f54a86704f0f39fe1c14c448ce54412a3dd322e60c9f1340dd7c82ed4 WatchSource:0}: Error finding container bcda9c4f54a86704f0f39fe1c14c448ce54412a3dd322e60c9f1340dd7c82ed4: Status 404 returned error can't find the container with id bcda9c4f54a86704f0f39fe1c14c448ce54412a3dd322e60c9f1340dd7c82ed4
Jul 17 10:17:18.887813 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:18.887774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/must-gather-7h7gh" event={"ID":"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba","Type":"ContainerStarted","Data":"bcda9c4f54a86704f0f39fe1c14c448ce54412a3dd322e60c9f1340dd7c82ed4"}
Jul 17 10:17:19.898777 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:19.897884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/must-gather-7h7gh" event={"ID":"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba","Type":"ContainerStarted","Data":"15ca6bc4f77a9ee7331609c5881259b14bed13b6d270f1f3d0b6cd0632bfd3b6"}
Jul 17 10:17:19.898777 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:19.897943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/must-gather-7h7gh" event={"ID":"1a3c3bfe-4f83-4a63-ae8f-d3434bea49ba","Type":"ContainerStarted","Data":"cd6e8b45e121f17035590d109aecf34619f828c7f20b2be1c88420f31efdebc3"}
Jul 17 10:17:19.923098 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:19.923040    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z5nh8/must-gather-7h7gh" podStartSLOduration=2.110223079 podStartE2EDuration="2.923017652s" totalImagesPullingTime="812.794573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:19.920004792 +0000 UTC m=+3553.311997410" watchObservedRunningTime="2026-07-17 10:17:19.923017652 +0000 UTC m=+3553.315010270"
Jul 17 10:17:20.841850 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:20.841809    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tl6xw_7849ad77-4297-4d63-97eb-4f60039e4a1b/global-pull-secret-syncer/0.log"
Jul 17 10:17:21.031963 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:21.031934    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nbvvs_bb36912c-324d-411b-a59b-9aa6c9bcd7b2/konnectivity-agent/0.log"
Jul 17 10:17:21.106824 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:21.106731    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-140.ec2.internal_986b8db4c072b1a4199cd963bf9e6ab5/haproxy/0.log"
Jul 17 10:17:26.853981 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:26.853935    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c4ntr_3de5614f-4875-48eb-b5aa-381bd6d167af/kuadrant-console-plugin/0.log"
Jul 17 10:17:28.643511 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.643483    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/alertmanager/0.log"
Jul 17 10:17:28.670488 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.670447    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/config-reloader/0.log"
Jul 17 10:17:28.696400 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.696376    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/kube-rbac-proxy-web/0.log"
Jul 17 10:17:28.724609 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.724529    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/kube-rbac-proxy/0.log"
Jul 17 10:17:28.750711 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.750680    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/kube-rbac-proxy-metric/0.log"
Jul 17 10:17:28.776550 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.776520    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/prom-label-proxy/0.log"
Jul 17 10:17:28.799723 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.799684    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f7f366ae-a012-4217-9b1e-ae395200fc9e/init-config-reloader/0.log"
Jul 17 10:17:28.964791 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:28.964717    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78b4458c64-qk8k2_853093a6-be68-4b1b-811d-7e061320b691/metrics-server/0.log"
Jul 17 10:17:29.108884 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.108851    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dh7wg_89bc3bc7-fc09-4617-9e2e-38db6e72e221/node-exporter/0.log"
Jul 17 10:17:29.133519 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.133477    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dh7wg_89bc3bc7-fc09-4617-9e2e-38db6e72e221/kube-rbac-proxy/0.log"
Jul 17 10:17:29.166948 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.166919    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dh7wg_89bc3bc7-fc09-4617-9e2e-38db6e72e221/init-textfile/0.log"
Jul 17 10:17:29.275111 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.275083    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zk549_0850f735-6a1a-46a1-8b5e-cdce30a3b214/kube-rbac-proxy-main/0.log"
Jul 17 10:17:29.300056 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.300026    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zk549_0850f735-6a1a-46a1-8b5e-cdce30a3b214/kube-rbac-proxy-self/0.log"
Jul 17 10:17:29.335397 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.335371    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zk549_0850f735-6a1a-46a1-8b5e-cdce30a3b214/openshift-state-metrics/0.log"
Jul 17 10:17:29.573449 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.573376    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-k5srx_a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0/prometheus-operator/0.log"
Jul 17 10:17:29.592999 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.592968    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-k5srx_a4dcc94f-1f40-472a-ad9a-4d7e658a4aa0/kube-rbac-proxy/0.log"
Jul 17 10:17:29.653293 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.653265    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7656dbdc6c-dg6l4_a84b09f9-6fab-4186-b7fb-59c8e17d6bd0/telemeter-client/0.log"
Jul 17 10:17:29.675874 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.675848    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7656dbdc6c-dg6l4_a84b09f9-6fab-4186-b7fb-59c8e17d6bd0/reload/0.log"
Jul 17 10:17:29.701026 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:29.700991    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7656dbdc6c-dg6l4_a84b09f9-6fab-4186-b7fb-59c8e17d6bd0/kube-rbac-proxy/0.log"
Jul 17 10:17:30.546499 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.546459    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"]
Jul 17 10:17:30.554181 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.554148    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.561560 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.561537    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z5nh8\"/\"default-dockercfg-qnv6m\""
Jul 17 10:17:30.570808 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.570771    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"]
Jul 17 10:17:30.697816 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.697778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-proc\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.698428 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.698403    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-sys\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.698711 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.698685    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-lib-modules\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.698871 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.698848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7ttz\" (UniqueName: \"kubernetes.io/projected/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-kube-api-access-r7ttz\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.699018 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.698959    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-podres\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.799987 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.799882    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-podres\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.799987 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.799967    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-proc\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-sys\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-lib-modules\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-podres\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-proc\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7ttz\" (UniqueName: \"kubernetes.io/projected/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-kube-api-access-r7ttz\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-sys\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.800233 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.800214    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-lib-modules\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.813519 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.813473    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7ttz\" (UniqueName: \"kubernetes.io/projected/9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed-kube-api-access-r7ttz\") pod \"perf-node-gather-daemonset-5sj6k\" (UID: \"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:30.868549 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:30.868517    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:31.095700 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.095672    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"]
Jul 17 10:17:31.741919 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.741878    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/1.log"
Jul 17 10:17:31.748039 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.748016    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-rpjth_b3528b82-4841-475a-9e5b-5d94d683ac5c/console-operator/2.log"
Jul 17 10:17:31.986113 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.986072    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k" event={"ID":"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed","Type":"ContainerStarted","Data":"a3eb2c90f262a1c63fc8eda780eec507ee7b52e0434075c2d408809f52f8b56f"}
Jul 17 10:17:31.986335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.986142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k" event={"ID":"9e8491c2-1c9e-4fed-bf4b-11e7cc5f22ed","Type":"ContainerStarted","Data":"256fd88446a2066a4676c15e9959337b31c0e651ad69104f9197eba08449fac4"}
Jul 17 10:17:31.986335 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:31.986202    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:32.010572 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:32.010489    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k" podStartSLOduration=2.010471768 podStartE2EDuration="2.010471768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:32.008574297 +0000 UTC m=+3565.400566927" watchObservedRunningTime="2026-07-17 10:17:32.010471768 +0000 UTC m=+3565.402464389"
Jul 17 10:17:34.168590 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:34.168553    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4jtlg_cc8a050f-5f64-4c67-b0a6-93d5877df02d/dns/0.log"
Jul 17 10:17:34.197348 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:34.197324    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4jtlg_cc8a050f-5f64-4c67-b0a6-93d5877df02d/kube-rbac-proxy/0.log"
Jul 17 10:17:34.418100 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:34.418071    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zkm7q_c8684fe2-2823-4aa4-bbbd-1ebcc2239523/dns-node-resolver/0.log"
Jul 17 10:17:35.039480 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:35.039450    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5wq6g_5da12a08-c074-4cad-b523-fffdc5fa3002/node-ca/0.log"
Jul 17 10:17:36.133446 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:36.133418    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45pw68_ce89b7b3-a842-4f44-894f-203708d84729/istio-proxy/0.log"
Jul 17 10:17:36.406311 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:36.406202    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-qznjf_8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a/kube-auth-proxy/0.log"
Jul 17 10:17:37.309379 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:37.309349    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-46ht7_234e65f8-8a04-40a6-bbcc-4f45ea67d6c4/serve-healthcheck-canary/0.log"
Jul 17 10:17:37.968242 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:37.968211    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ls2cv_c97aa36f-dada-4dba-ab4e-d7e3e6c18606/kube-rbac-proxy/0.log"
Jul 17 10:17:37.992814 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:37.992782    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ls2cv_c97aa36f-dada-4dba-ab4e-d7e3e6c18606/exporter/0.log"
Jul 17 10:17:38.006866 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:38.006839    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-5sj6k"
Jul 17 10:17:38.022567 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:38.022544    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ls2cv_c97aa36f-dada-4dba-ab4e-d7e3e6c18606/extractor/0.log"
Jul 17 10:17:41.059507 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:41.059480    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45pw68_ce89b7b3-a842-4f44-894f-203708d84729/istio-proxy/0.log"
Jul 17 10:17:41.314504 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:41.314404    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-rhjtm_718156a7-f6a4-4e41-8b63-56c317205082/manager/0.log"
Jul 17 10:17:41.328788 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:41.328752    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-qznjf_8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a/kube-auth-proxy/0.log"
Jul 17 10:17:41.335435 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:41.335406    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-rhjtm_718156a7-f6a4-4e41-8b63-56c317205082/copy-manifests/0.log"
Jul 17 10:17:43.014834 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:43.014767    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dd756d967-fzkpn_a63cdf7c-8b23-46a2-82f7-d930df916630/manager/0.log"
Jul 17 10:17:43.024339 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:43.024311    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45pw68_ce89b7b3-a842-4f44-894f-203708d84729/istio-proxy/0.log"
Jul 17 10:17:43.056356 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:43.056330    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-bk2r2_30bf22d5-0cd5-4d3a-81ab-7f0df9a55c5d/openshift-lws-operator/0.log"
Jul 17 10:17:43.300673 ip-10-0-139-140 kubenswrapper[2575]: I0717 10:17:43.300603    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-qznjf_8f5157c3-9d5f-4230-a0ae-fbd57b2c7e4a/kube-auth-proxy/0.log"