Jul 22 01:27:59.996953 ip-10-0-140-155 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 01:28:00.381161 ip-10-0-140-155 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 01:28:00.381161 ip-10-0-140-155 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 01:28:00.381161 ip-10-0-140-155 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 01:28:00.381161 ip-10-0-140-155 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 01:28:00.381161 ip-10-0-140-155 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 01:28:00.382591 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.382509    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 01:28:00.386398 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386382    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.386398 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386398    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386403    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386407    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386412    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386416    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386419    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386421    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386424    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386427    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386429    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386432    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386434    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386437    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386439    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386442    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386445    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386447    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386450    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386453    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.386458 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386456    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386459    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386462    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386465    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386468    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386471    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386474    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386477    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386479    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386482    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386484    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386486    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386489    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386492    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386494    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386497    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386499    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386502    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386505    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386507    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.386912 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386510    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386512    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386515    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386517    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386519    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386522    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386525    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386527    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386530    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386532    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386535    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386537    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386539    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386542    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386545    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386547    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386550    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386553    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386555    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386557    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.387371 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386560    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386564    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386568    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386570    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386573    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386577    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386579    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386582    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386584    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386587    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386589    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386592    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386595    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386597    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386599    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386602    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386604    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386607    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386610    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386612    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.387844 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386615    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386618    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386622    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386625    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386628    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386632    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386635    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386638    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386640    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386644    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386647    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386650    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386652    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386654    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386657    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386659    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386662    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386664    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.386667    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.388293 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387387    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387395    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387398    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387401    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387403    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387406    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387409    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387412    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387414    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387417    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387420    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387423    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387425    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387428    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387430    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387434    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387438    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387441    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387445    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.388772 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387448    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387450    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387453    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387456    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387458    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387461    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387463    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387466    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387468    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387470    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387473    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387475    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387478    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387480    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387483    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387485    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387488    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387490    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387493    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387497    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.389228 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387499    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387502    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387504    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387506    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387509    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387512    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387514    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387517    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387519    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387522    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387524    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387527    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387529    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387532    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387534    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387537    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387541    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387544    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387547    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.389676 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387549    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387553    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387555    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387558    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387560    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387562    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387565    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387568    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387570    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387572    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387576    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387580    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387583    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387587    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387589    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387592    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387595    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387598    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387601    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387604    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.390127 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387606    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387608    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387611    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387614    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387616    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387619    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387621    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387624    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387627    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387629    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387632    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387634    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387637    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387639    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387642    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387645    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387647    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387650    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387652    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387655    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.390584 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.387657    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388059    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388065    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388068    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388070    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388078    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388081    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388084    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388086    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388089    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388091    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388093    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388096    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388098    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388101    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388104    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388107    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388109    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388111    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.391090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388114    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388117    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388119    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388122    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388125    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388127    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388130    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388133    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388136    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388138    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388140    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388143    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388145    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388148    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388150    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388153    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388155    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388158    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388160    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388163    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.391519 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388165    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388168    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388170    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388173    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388177    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388181    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388184    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388187    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388190    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388194    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388196    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388199    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388203    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388206    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388209    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388212    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388214    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388217    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388219    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388223    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.391987 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388225    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388228    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388230    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388233    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388235    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388238    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388240    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388243    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388245    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388248    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388250    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388253    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388261    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388264    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388266    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388268    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388271    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388274    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388276    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388280    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.392482 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388282    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388285    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388287    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388290    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388293    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388296    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388298    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388301    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388303    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388306    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388308    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388313    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388316    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388319    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388321    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388324    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388326    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388329    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388331    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388334    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.392962 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.388336    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388416    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388429    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388437    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388441    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388446    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388449    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388454    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388458    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388462    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388465    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388468    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388472    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388474    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388478    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388481    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388483    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388486    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388489    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388493    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388496    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388499    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388502    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388510    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 01:28:00.393458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388515    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388518    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388521    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388525    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388527    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388530    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388533    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388536    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388539    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388543    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388546    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388549    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388552    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388554    2560 flags.go:64] FLAG: --enable-server="true"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388558    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388563    2560 flags.go:64] FLAG: --event-burst="100"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388566    2560 flags.go:64] FLAG: --event-qps="50"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388569    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388572    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388575    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388579    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388582    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388585    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388588    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388591    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 01:28:00.394054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388594    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388597    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388600    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388602    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388605    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388608    2560 flags.go:64] FLAG: --feature-gates=""
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388613    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388616    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388619    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388622    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388625    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388628    2560 flags.go:64] FLAG: --help="false"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388631    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-140-155.ec2.internal"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388634    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388637    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388640    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388644    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388647    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388650    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388652    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388655    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388658    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388661    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388667    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 01:28:00.394643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388670    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388673    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388675    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388678    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388681    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388684    2560 flags.go:64] FLAG: --lock-file=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388700    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388703    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388706    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388712    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388714    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388717    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388720    2560 flags.go:64] FLAG: --logging-format="text"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388723    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388728    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388731    2560 flags.go:64] FLAG: --manifest-url=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388734    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388738    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388741    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388745    2560 flags.go:64] FLAG: --max-pods="110"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388748    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388751    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388754    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388757    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388760    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 01:28:00.395232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388762    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388765    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388773    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388775    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388779    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388782    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388786    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388792    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388795    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388798    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388801    2560 flags.go:64] FLAG: --port="10250"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388810    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388813    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0af1574f156325808"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388817    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388820    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388823    2560 flags.go:64] FLAG: --register-node="true"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388826    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388829    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388832    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388835    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388838    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388842    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388845    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388848    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388850    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 01:28:00.395840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388853    2560 flags.go:64] FLAG: --runonce="false"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388856    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388859    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388862    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388865    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388868    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388871    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388874    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388877    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388879    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388882    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388885    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388888    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388891    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388896    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388899    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388904    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388907    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388910    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388913    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388916    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388919    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388922    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388924    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388927    2560 flags.go:64] FLAG: --v="2"
Jul 22 01:28:00.396482 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388931    2560 flags.go:64] FLAG: --version="false"
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388935    2560 flags.go:64] FLAG: --vmodule=""
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388939    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.388942    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389096    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389101    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389104    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389108    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389112    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389115    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389118    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389121    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389123    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389126    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389128    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389131    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389133    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389136    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389138    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389141    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.397090 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389144    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389147    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389150    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389153    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389156    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389158    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389161    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389163    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389166    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389168    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389171    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389173    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389176    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389179    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389182    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389184    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389187    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389190    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389192    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389195    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.397588 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389197    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389200    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389202    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389205    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389207    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389210    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389212    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389215    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389217    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389219    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389222    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389224    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389227    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389231    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389235    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389240    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389243    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389246    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389249    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.398087 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389252    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389255    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389257    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389260    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389262    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389265    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389267    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389269    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389272    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389274    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389277    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389279    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389281    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389284    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389286    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389289    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389291    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389294    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389296    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389299    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.398538 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389301    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389303    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389306    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389308    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389310    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389313    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389317    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389319    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389322    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389324    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389327    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389330    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389334    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389336    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389338    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389341    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389344    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389348    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389350    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.399040 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389353    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389355    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389358    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389361    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.389363    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.389779    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.396255    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.396270    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.396287    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.396292    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396348    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396352    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396356    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396359    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396362    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.399494 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396365    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396368    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396371    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396373    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396376    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396379    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396382    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396385    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396387    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396390    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396392    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396395    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396397    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396401    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396405    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396407    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396410    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396413    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396415    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396418    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.399875 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396420    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396423    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396425    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396427    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396432    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396434    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396437    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396439    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396443    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396445    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396448    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396450    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396452    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396455    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396457    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396459    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396462    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396464    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396467    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396470    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.400349 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396472    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396474    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396477    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396479    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396482    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396485    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396488    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396490    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396493    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396496    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396498    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396501    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396504    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396507    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396509    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396512    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396514    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396517    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396520    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.400838 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396522    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396526    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396529    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396531    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396534    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396537    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396540    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396542    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396545    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396547    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396550    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396553    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396555    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396559    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396563    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396566    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396569    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396572    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396575    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.401270 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396578    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396581    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396583    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396586    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396588    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396591    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396593    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396595    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396598    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396600    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396603    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396605    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396608    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396611    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396613    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.401866 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396615    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.396621    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396761    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396767    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396770    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396773    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396776    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396779    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396781    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396784    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396786    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396790    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396793    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396796    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 01:28:00.402225 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396798    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396800    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396811    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396813    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396817    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396820    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396823    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396826    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396829    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396831    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396834    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396837    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396839    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396842    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396845    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396848    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396851    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396853    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396856    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 01:28:00.402548 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396858    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396860    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396863    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396865    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396867    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396870    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396872    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396875    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396877    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396879    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396882    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396884    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396887    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396889    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396891    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396894    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396897    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396899    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396902    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396904    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 01:28:00.403005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396906    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396909    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396911    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396914    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396916    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396918    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396921    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396924    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396926    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396928    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396931    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396934    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396936    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396938    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396940    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396943    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396945    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396947    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396950    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396952    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 01:28:00.403537 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396954    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396956    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396959    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396961    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396964    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396966    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396968    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396971    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396974    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396976    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396978    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396980    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396983    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396985    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396988    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396990    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396994    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396996    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.396999    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397002    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 01:28:00.404218 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397004    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397007    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397010    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397013    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397015    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397018    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397020    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:00.397022    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.397027    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.397132    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.399152    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.400387    2560 server.go:1019] "Starting client certificate rotation"
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.400483    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 01:28:00.404817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.401092    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 01:28:00.405157 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.402542    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 01:28:00.406213 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.406196    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 01:28:00.414097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.414079    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 22 01:28:00.418072 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.418059    2560 log.go:25] "Validated CRI v1 image API"
Jul 22 01:28:00.418818 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.418789    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 01:28:00.421295 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.421277    2560 fs.go:135] Filesystem UUIDs: map[60b3d590-1287-4ab9-a5ba-6cc7e9c6ff6c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e4b0838e-e334-4e19-87bb-6b0d53827a4f:/dev/nvme0n1p3]
Jul 22 01:28:00.421351 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.421295    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 01:28:00.426850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.426745    2560 manager.go:216] Machine: {Timestamp:2026-07-22 01:28:00.425340188 +0000 UTC m=+0.319462733 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196122 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22c3317f493f303b3116a34458f851 SystemUUID:ec22c331-7f49-3f30-3b31-16a34458f851 BootID:17b500ed-e1ec-48f7-81d8-397a1ac124d6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ec:ec:0b:5d:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ec:ec:0b:5d:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:81:c9:f7:9b:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 01:28:00.426850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.426844    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 01:28:00.426950 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.426943    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 01:28:00.429380 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429356    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 01:28:00.429507 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429382    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 01:28:00.429894 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429879    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 01:28:00.429973 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429962    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 01:28:00.430005 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429975    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 01:28:00.430005 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.429988    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 01:28:00.430554 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.430544    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 01:28:00.431664 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.431654    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 01:28:00.434363 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434348    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 01:28:00.434495 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434484    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 01:28:00.434549 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434522    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 01:28:00.434549 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434537    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 01:28:00.434634 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434557    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 01:28:00.434634 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434571    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 22 01:28:00.434634 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434583    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 01:28:00.434962 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434949    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 01:28:00.435031 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.434971    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 01:28:00.437004 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.436990    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 01:28:00.439049 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.439036    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 01:28:00.439096 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.439067    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 01:28:00.440062 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440050    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440066    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440072    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440078    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440089    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440097    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440106    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 01:28:00.440110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440112    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 01:28:00.440294 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440119    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 01:28:00.440294 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440133    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 01:28:00.440294 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440145    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 01:28:00.440294 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440150    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 01:28:00.440767 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440757    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 01:28:00.440798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.440769    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 01:28:00.443940 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.443927    2560 server.go:1268] "Started kubelet"
Jul 22 01:28:00.446926 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.446857    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 01:28:00.447020 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.446943    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 01:28:00.447261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.447237    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 01:28:00.447523 ip-10-0-140-155 systemd[1]: Started Kubernetes Kubelet.
Jul 22 01:28:00.447904 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.447874    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 01:28:00.448270 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.448135    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 01:28:00.448270 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.448253    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 01:28:00.448505 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.448483    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 01:28:00.450730 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.450714    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 01:28:00.454446 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.453473    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-155.ec2.internal.18c4785a094b9b0c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-155.ec2.internal,UID:ip-10-0-140-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-155.ec2.internal,},FirstTimestamp:2026-07-22 01:28:00.443898636 +0000 UTC m=+0.338021156,LastTimestamp:2026-07-22 01:28:00.443898636 +0000 UTC m=+0.338021156,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-155.ec2.internal,}"
Jul 22 01:28:00.454537 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.454506    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 01:28:00.455876 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.455864    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 01:28:00.455932 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.455881    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 01:28:00.456023 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.456004    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:00.456261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456232    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6rz54"
Jul 22 01:28:00.456469 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456442    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 01:28:00.456469 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456445    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 01:28:00.456583 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456471    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 01:28:00.456583 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456571    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 01:28:00.456583 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456580    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 01:28:00.456742 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456726    2560 factory.go:55] Registering systemd factory
Jul 22 01:28:00.456742 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456744    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 22 01:28:00.456963 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456949    2560 factory.go:153] Registering CRI-O factory
Jul 22 01:28:00.457030 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.456967    2560 factory.go:223] Registration of the crio container factory successfully
Jul 22 01:28:00.457030 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.457017    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 01:28:00.457116 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.457043    2560 factory.go:103] Registering Raw factory
Jul 22 01:28:00.457116 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.457057    2560 manager.go:1196] Started watching for new ooms in manager
Jul 22 01:28:00.457424 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.457412    2560 manager.go:349] Starting recovery of all containers
Jul 22 01:28:00.458379 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.458358    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 01:28:00.458789 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.458763    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 01:28:00.464442 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.464394    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 01:28:00.464727 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.464709    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6rz54"
Jul 22 01:28:00.467358 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.467333    2560 manager.go:354] Recovery completed
Jul 22 01:28:00.469285 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.469265    2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 22 01:28:00.473838 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.473825    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:00.475756 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.475740    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:00.475833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.475767    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:00.475833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.475785    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:00.476309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.476296    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 01:28:00.476309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.476308    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 01:28:00.476403 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.476322    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 01:28:00.476403 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.476346    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:00.477077 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477059    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:00.477165 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477085    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:00.477165 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477096    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:00.477433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477419    2560 policy_none.go:49] "None policy: Start"
Jul 22 01:28:00.477488 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477437    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 01:28:00.477488 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.477451    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 01:28:00.477803 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.477704    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-155.ec2.internal.18c4785a0b31b2b2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-155.ec2.internal,UID:ip-10-0-140-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-155.ec2.internal,},FirstTimestamp:2026-07-22 01:28:00.475755186 +0000 UTC m=+0.369877712,LastTimestamp:2026-07-22 01:28:00.475755186 +0000 UTC m=+0.369877712,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-155.ec2.internal,}"
Jul 22 01:28:00.478352 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.478338    2560 policy_none.go:47] "Start"
Jul 22 01:28:00.516273 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516258    2560 manager.go:337] "Starting Device Plugin manager"
Jul 22 01:28:00.516359 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.516296    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 01:28:00.516359 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516306    2560 server.go:87] "Starting device plugin registration server"
Jul 22 01:28:00.516568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516556    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 01:28:00.516789 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516572    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 01:28:00.516877 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516860    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 01:28:00.517081 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516946    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 01:28:00.517081 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.516958    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 01:28:00.517356 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.517337    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 01:28:00.517424 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.517390    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:00.617124 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.617085    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:00.618516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.618066    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:00.618516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.618093    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:00.618516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.618104    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:00.618516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.618134    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-155.ec2.internal"
Jul 22 01:28:00.626260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.626244    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-155.ec2.internal"
Jul 22 01:28:00.626337 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.626266    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-155.ec2.internal\": node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:00.638758 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.638686    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 01:28:00.638758 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.638736    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 01:28:00.638758 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.638756    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 01:28:00.638962 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.638784    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 01:28:00.639722 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.639705    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:00.640679 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.640658    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:00.738915 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.738852    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"]
Jul 22 01:28:00.739075 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.738962    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:00.740275 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:00.740256    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:00.740678 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.740654    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:00.740786 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.740704    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:00.740786 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:00.740716    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:02.087706 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.087692    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:02.087844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.087830    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.087910 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.087858    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:02.088477 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088459    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:02.088552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088490    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:02.088552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088500    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:02.088552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088459    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:02.088637 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088560    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:02.088637 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.088571    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:02.089734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.089717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.089825 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.089741    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 01:28:02.090325 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.090309    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 01:28:02.090427 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.090329    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 01:28:02.090427 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.090338    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeHasSufficientPID"
Jul 22 01:28:02.103635 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.103612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.103720 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.103637    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.103720 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.103652    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a5bc24e7d34684078d3262a35f51e63-config\") pod \"kube-apiserver-proxy-ip-10-0-140-155.ec2.internal\" (UID: \"3a5bc24e7d34684078d3262a35f51e63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.106968 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.106950    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-155.ec2.internal\" not found" node="ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.110813 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.110798    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-155.ec2.internal\" not found" node="ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.186016 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.185996    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:02.204651 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204627    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.204733 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204662    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.204733 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204680    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a5bc24e7d34684078d3262a35f51e63-config\") pod \"kube-apiserver-proxy-ip-10-0-140-155.ec2.internal\" (UID: \"3a5bc24e7d34684078d3262a35f51e63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.204799 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204726    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.204799 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/30e58754db9f26f88af6667bbe43cbd9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal\" (UID: \"30e58754db9f26f88af6667bbe43cbd9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.204799 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.204789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3a5bc24e7d34684078d3262a35f51e63-config\") pod \"kube-apiserver-proxy-ip-10-0-140-155.ec2.internal\" (UID: \"3a5bc24e7d34684078d3262a35f51e63\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.286849 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.286761    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:02.387340 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.387298    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:02.410756 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.410732    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.413337 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.413311    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.488188 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.488151    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:02.588714 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.588645    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-155.ec2.internal\" not found"
Jul 22 01:28:02.599196 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.599176    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:02.602144 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.602128    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.613009 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.612988    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 01:28:02.614949 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.614935    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"
Jul 22 01:28:02.623509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.623490    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 01:28:02.662739 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.662714    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:02.745952 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.745922    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 01:28:02.746385 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.746082    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 01:28:02.746385 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.746096    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 01:28:02.746385 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.746120    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 01:28:02.781220 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.781197    2560 apiserver.go:52] "Watching apiserver"
Jul 22 01:28:02.797733 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.797713    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 01:28:02.798733 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.798702    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg","openshift-dns/node-resolver-ngqhh","openshift-image-registry/node-ca-b88qd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal","openshift-multus/multus-additional-cni-plugins-xx4b8","openshift-multus/multus-xclt9","kube-system/global-pull-secret-syncer-8t8r6","kube-system/konnectivity-agent-lh9bc","openshift-cluster-node-tuning-operator/tuned-tx4l4","openshift-multus/network-metrics-daemon-gmzh4","openshift-network-diagnostics/network-check-target-7jxjz","openshift-network-operator/iptables-alerter-g2v4g","openshift-ovn-kubernetes/ovnkube-node-rdrdw","kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal"]
Jul 22 01:28:02.800500 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.800476    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.801695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.801676    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.801790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.801742    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 01:28:02.802503 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.802481    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzgpn\""
Jul 22 01:28:02.802503 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.802489    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.802715 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.802700    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.802841 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.802826    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.803909 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.803891    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:02.804000 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.803978    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:02.804742 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804727    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.804812 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804744    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 01:28:02.804812 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804800    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 01:28:02.804981 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804962    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 01:28:02.804981 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804965    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 01:28:02.805088 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805027    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.805088 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z9w8m\""
Jul 22 01:28:02.805179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805082    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.805179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 01:28:02.805179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805028    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.805179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.804962    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-frnwp\""
Jul 22 01:28:02.805309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.805293    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.806327 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.806314    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:02.806385 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.806370    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:02.807360 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807347    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:02.807758 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807743    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-89vjd\""
Jul 22 01:28:02.807845 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807827    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-systemd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.807904 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-lib-modules\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.807904 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807889    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-tmp\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.807994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807919    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-systemd-units\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.807994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-modprobe-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.807994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807970    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-host\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.807993    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-systemd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808016    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-slash\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-netns\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808075    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808078    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-bin\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808059    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 01:28:02.808106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-netd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-var-lib-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808221    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-log-socket\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808235    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-iptables-alerter-script\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llg42\" (UniqueName: \"kubernetes.io/projected/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-kube-api-access-llg42\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-kubernetes\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-sys\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808381    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808412    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-env-overrides\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808428    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovn-node-metrics-cert\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-script-lib\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlfsd\" (UniqueName: \"kubernetes.io/projected/97c8d377-9f37-4601-b111-36d80763e116-kube-api-access-dlfsd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808470    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-config\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-var-lib-kubelet\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-etc-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-host-slash\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808589    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-conf\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-run\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808629    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-etc-tuned\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-kubelet\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.808672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808672    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vfs\" (UniqueName: \"kubernetes.io/projected/9c97dcd2-f082-47fd-952b-e78a90c6375f-kube-api-access-22vfs\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.809164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-ovn\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.809164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808716    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-node-log\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.809164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysconfig\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.809164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.808888    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.809489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.809476    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 01:28:02.809821 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.809805    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 01:28:02.809862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.809826    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wrb6w\""
Jul 22 01:28:02.810067 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.810051    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.810751 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.810737    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 01:28:02.811037 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.811023    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k7k26\""
Jul 22 01:28:02.811368 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.811353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:02.811540 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:02.811520    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:02.811594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.811529    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 01:28:02.811893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.811879    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 01:28:02.812282 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.812269    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.812335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.812307    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-998sh\""
Jul 22 01:28:02.812378 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.812368    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.812817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.812802    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:02.813976 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.813962    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 01:28:02.813976 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.813936    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 01:23:01 +0000 UTC" deadline="2028-04-30 22:08:36.595822306 +0000 UTC"
Jul 22 01:28:02.814072 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.813985    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15572h40m33.781841879s"
Jul 22 01:28:02.814237 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.814223    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:02.814845 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.814829    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 01:28:02.814957 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.814945    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.815151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.815128    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.815191 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.815150    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-244z9\""
Jul 22 01:28:02.816431 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.816417    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 01:28:02.816468 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.816445    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 01:28:02.816642 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.816631    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-clm7z\""
Jul 22 01:28:02.833552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.833530    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfknr"
Jul 22 01:28:02.839573 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.839524    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hfknr"
Jul 22 01:28:02.903580 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.903561    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 01:28:02.909232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909277 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909240    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-etc-tuned\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909277 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909256    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vfs\" (UniqueName: \"kubernetes.io/projected/9c97dcd2-f082-47fd-952b-e78a90c6375f-kube-api-access-22vfs\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909349 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4m99\" (UniqueName: \"kubernetes.io/projected/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kube-api-access-q4m99\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.909349 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909299    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcqgr\" (UniqueName: \"kubernetes.io/projected/c6ef6ede-818c-4d00-a20a-6d197988f37c-kube-api-access-lcqgr\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:02.909443 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909429    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-multus-certs\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.909494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909463    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-agent-certs\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:02.909494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-dbus\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-os-release\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909524    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-ovn\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysconfig\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909553    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-tmp\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-systemd-units\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909589    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-ovn\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909601    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-systemd-units\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-slash\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909613    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysconfig\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909635    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-slash\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909638    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-bin\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909656    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-netd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-bin\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909670    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-iptables-alerter-script\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llg42\" (UniqueName: \"kubernetes.io/projected/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-kube-api-access-llg42\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-cni-netd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909701    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909719    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909753    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909756    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c43001b2-2152-4107-8421-cf2cab3e7e7b-serviceca\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909779    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-cnibin\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lwp\" (UniqueName: \"kubernetes.io/projected/fe14439e-c018-434a-b8f7-cf84431ae73b-kube-api-access-67lwp\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-kubernetes\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909850    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.909906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909895    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-kubernetes\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909915    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d44rc\" (UniqueName: \"kubernetes.io/projected/6af796ea-73a6-41c9-a711-e128aacb9a60-kube-api-access-d44rc\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-system-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-daemon-config\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-script-lib\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-registration-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.909992    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6ef6ede-818c-4d00-a20a-6d197988f37c-hosts-file\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910008    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6ef6ede-818c-4d00-a20a-6d197988f37c-tmp-dir\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910065    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-netns\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910079    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlfsd\" (UniqueName: \"kubernetes.io/projected/97c8d377-9f37-4601-b111-36d80763e116-kube-api-access-dlfsd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-socket-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-sys-fs\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-bin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.910740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910255    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-kubelet\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-etc-kubernetes\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-konnectivity-ca\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910334    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-etc-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910357    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-host-slash\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910372    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-etc-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910378    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910419    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-k8s-cni-cncf-io\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910432    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-host-slash\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-conf\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-run\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910516    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-kubelet\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-socket-dir-parent\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-multus\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910607    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-run\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-script-lib\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-sysctl-conf\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-kubelet\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.911455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-hostroot\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910694    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-kubelet-config\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-node-log\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-systemd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910770    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-lib-modules\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-netns\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-env-overrides\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910832    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-node-log\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-systemd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910847    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-lib-modules\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-netns\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-modprobe-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-host\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.910979    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-etc-modprobe-d\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-systemd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-host\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-systemd\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg72f\" (UniqueName: \"kubernetes.io/projected/c43001b2-2152-4107-8421-cf2cab3e7e7b-kube-api-access-gg72f\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911120    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-conf-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-var-lib-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-var-lib-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911192    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-run-openvswitch\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911228    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-log-socket\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-etc-selinux\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911271    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-log-socket\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-env-overrides\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-os-release\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911340    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-sys\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911383    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovn-node-metrics-cert\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.912632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-device-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911426    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c97dcd2-f082-47fd-952b-e78a90c6375f-host-run-ovn-kubernetes\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-iptables-alerter-script\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-sys\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911442    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911467    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-config\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ssmm\" (UniqueName: \"kubernetes.io/projected/1545bd2f-f722-4fef-9083-4977fed190d7-kube-api-access-9ssmm\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911530    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-var-lib-kubelet\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911550    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c43001b2-2152-4107-8421-cf2cab3e7e7b-host\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911561    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/97c8d377-9f37-4601-b111-36d80763e116-var-lib-kubelet\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-cnibin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-cni-binary-copy\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:02.913095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.911911    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovnkube-config\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.913598 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.913461    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-tmp\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.913598 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.913517    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/97c8d377-9f37-4601-b111-36d80763e116-etc-tuned\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.913681 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.913591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c97dcd2-f082-47fd-952b-e78a90c6375f-ovn-node-metrics-cert\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.916693 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.916666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vfs\" (UniqueName: \"kubernetes.io/projected/9c97dcd2-f082-47fd-952b-e78a90c6375f-kube-api-access-22vfs\") pod \"ovnkube-node-rdrdw\" (UID: \"9c97dcd2-f082-47fd-952b-e78a90c6375f\") " pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:02.917424 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.917405    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlfsd\" (UniqueName: \"kubernetes.io/projected/97c8d377-9f37-4601-b111-36d80763e116-kube-api-access-dlfsd\") pod \"tuned-tx4l4\" (UID: \"97c8d377-9f37-4601-b111-36d80763e116\") " pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:02.917805 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:02.917787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llg42\" (UniqueName: \"kubernetes.io/projected/3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf-kube-api-access-llg42\") pod \"iptables-alerter-g2v4g\" (UID: \"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf\") " pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:03.011926 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.011895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-registration-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.011926 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.011929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6ef6ede-818c-4d00-a20a-6d197988f37c-hosts-file\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.011955    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6ef6ede-818c-4d00-a20a-6d197988f37c-tmp-dir\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.011975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.011995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012015    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012018    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-registration-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c6ef6ede-818c-4d00-a20a-6d197988f37c-hosts-file\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-netns\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-netns\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012092    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-socket-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-sys-fs\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012144    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-bin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.012158    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:03.012171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-kubelet\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012189    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-etc-kubernetes\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012200    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-sys-fs\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012211    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-konnectivity-ca\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012237    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-k8s-cni-cncf-io\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-socket-dir-parent\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012281    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-multus\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-kubelet\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-hostroot\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-socket-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012335    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-kubelet-config\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012347    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-etc-kubernetes\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012350    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-hostroot\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012380    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6ef6ede-818c-4d00-a20a-6d197988f37c-tmp-dir\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012411    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-bin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012422    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-socket-dir-parent\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.012807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012461    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-k8s-cni-cncf-io\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.012465    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:03.512439069 +0000 UTC m=+2.060857187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012471    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-kubelet-config\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012506    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-var-lib-cni-multus\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012545    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg72f\" (UniqueName: \"kubernetes.io/projected/c43001b2-2152-4107-8421-cf2cab3e7e7b-kube-api-access-gg72f\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-conf-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012591    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-etc-selinux\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012604    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012644    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-os-release\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012659    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-conf-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012682    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-device-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.013650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012720    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-os-release\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012728    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ssmm\" (UniqueName: \"kubernetes.io/projected/1545bd2f-f722-4fef-9083-4977fed190d7-kube-api-access-9ssmm\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c43001b2-2152-4107-8421-cf2cab3e7e7b-host\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012764    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-device-dir\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2e491c8-6fc3-4102-aa46-99f673c17ae0-etc-selinux\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012775    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-cnibin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012787    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-system-cni-dir\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-cni-binary-copy\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012818    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-cnibin\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012836    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4m99\" (UniqueName: \"kubernetes.io/projected/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kube-api-access-q4m99\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcqgr\" (UniqueName: \"kubernetes.io/projected/c6ef6ede-818c-4d00-a20a-6d197988f37c-kube-api-access-lcqgr\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-multus-certs\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-agent-certs\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-konnectivity-ca\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012934    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-dbus\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.012966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-os-release\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c43001b2-2152-4107-8421-cf2cab3e7e7b-host\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.014204 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013007    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013052    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c43001b2-2152-4107-8421-cf2cab3e7e7b-serviceca\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013057    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a15a6203-7643-4de5-b179-d107b53699f1-dbus\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.013066    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013103    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.013122    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:03.513103804 +0000 UTC m=+2.061521905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-host-run-multus-certs\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013146    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-os-release\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013185    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-cnibin\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013235    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lwp\" (UniqueName: \"kubernetes.io/projected/fe14439e-c018-434a-b8f7-cf84431ae73b-kube-api-access-67lwp\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013258    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-cni-binary-copy\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013261    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1545bd2f-f722-4fef-9083-4977fed190d7-cnibin\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d44rc\" (UniqueName: \"kubernetes.io/projected/6af796ea-73a6-41c9-a711-e128aacb9a60-kube-api-access-d44rc\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013320    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013325    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-system-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.014695 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-daemon-config\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.015154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fe14439e-c018-434a-b8f7-cf84431ae73b-system-cni-dir\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.015154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013584    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c43001b2-2152-4107-8421-cf2cab3e7e7b-serviceca\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.015154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fe14439e-c018-434a-b8f7-cf84431ae73b-multus-daemon-config\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.015154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.013806    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1545bd2f-f722-4fef-9083-4977fed190d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.015300 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.015152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc3b75e8-d9b2-42d6-8947-70ea7629fea8-agent-certs\") pod \"konnectivity-agent-lh9bc\" (UID: \"fc3b75e8-d9b2-42d6-8947-70ea7629fea8\") " pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:03.024485 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.024466    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:03.024485 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.024485    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:03.024574 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.024494    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:03.024574 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.024547    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:03.524534974 +0000 UTC m=+2.072953108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:03.026769 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.026746    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ssmm\" (UniqueName: \"kubernetes.io/projected/1545bd2f-f722-4fef-9083-4977fed190d7-kube-api-access-9ssmm\") pod \"multus-additional-cni-plugins-xx4b8\" (UID: \"1545bd2f-f722-4fef-9083-4977fed190d7\") " pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.027899 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.027882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d44rc\" (UniqueName: \"kubernetes.io/projected/6af796ea-73a6-41c9-a711-e128aacb9a60-kube-api-access-d44rc\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:03.028201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.028181    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg72f\" (UniqueName: \"kubernetes.io/projected/c43001b2-2152-4107-8421-cf2cab3e7e7b-kube-api-access-gg72f\") pod \"node-ca-b88qd\" (UID: \"c43001b2-2152-4107-8421-cf2cab3e7e7b\") " pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.028286 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.028187    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcqgr\" (UniqueName: \"kubernetes.io/projected/c6ef6ede-818c-4d00-a20a-6d197988f37c-kube-api-access-lcqgr\") pod \"node-resolver-ngqhh\" (UID: \"c6ef6ede-818c-4d00-a20a-6d197988f37c\") " pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.028286 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.028217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4m99\" (UniqueName: \"kubernetes.io/projected/f2e491c8-6fc3-4102-aa46-99f673c17ae0-kube-api-access-q4m99\") pod \"aws-ebs-csi-driver-node-8rntg\" (UID: \"f2e491c8-6fc3-4102-aa46-99f673c17ae0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.028456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.028440    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lwp\" (UniqueName: \"kubernetes.io/projected/fe14439e-c018-434a-b8f7-cf84431ae73b-kube-api-access-67lwp\") pod \"multus-xclt9\" (UID: \"fe14439e-c018-434a-b8f7-cf84431ae73b\") " pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.083106 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.083062    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5bc24e7d34684078d3262a35f51e63.slice/crio-961c28257b1a3725a9b2cb549059556450d36866099f4f934b0bf42830a24352 WatchSource:0}: Error finding container 961c28257b1a3725a9b2cb549059556450d36866099f4f934b0bf42830a24352: Status 404 returned error can't find the container with id 961c28257b1a3725a9b2cb549059556450d36866099f4f934b0bf42830a24352
Jul 22 01:28:03.083505 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.083485    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e58754db9f26f88af6667bbe43cbd9.slice/crio-7c25fa59ea4d0362614a55d63fc4bbb818364bd9992cf1d01645758b5ceda4f5 WatchSource:0}: Error finding container 7c25fa59ea4d0362614a55d63fc4bbb818364bd9992cf1d01645758b5ceda4f5: Status 404 returned error can't find the container with id 7c25fa59ea4d0362614a55d63fc4bbb818364bd9992cf1d01645758b5ceda4f5
Jul 22 01:28:03.088976 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.088960    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 01:28:03.146959 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.146925    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tx4l4"
Jul 22 01:28:03.155896 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.155871    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c8d377_9f37_4601_b111_36d80763e116.slice/crio-26a93bb555d165c3acaaaffc2b66c266573c4051e1a71600aceb8b123415ca34 WatchSource:0}: Error finding container 26a93bb555d165c3acaaaffc2b66c266573c4051e1a71600aceb8b123415ca34: Status 404 returned error can't find the container with id 26a93bb555d165c3acaaaffc2b66c266573c4051e1a71600aceb8b123415ca34
Jul 22 01:28:03.165813 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.165795    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:03.173982 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.173958    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c97dcd2_f082_47fd_952b_e78a90c6375f.slice/crio-40a65d9d5049faab5ccc50cd0bded0c2b50e0d310999646086c7c91828be1e2c WatchSource:0}: Error finding container 40a65d9d5049faab5ccc50cd0bded0c2b50e0d310999646086c7c91828be1e2c: Status 404 returned error can't find the container with id 40a65d9d5049faab5ccc50cd0bded0c2b50e0d310999646086c7c91828be1e2c
Jul 22 01:28:03.183551 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.183534    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g2v4g"
Jul 22 01:28:03.190818 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.190798    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3780a4_44d1_47ec_801e_8a3a8b7a2cbf.slice/crio-a3a960ffd25a6397e364926562763526f0d6b5b1b74bc779de97a269c64931ca WatchSource:0}: Error finding container a3a960ffd25a6397e364926562763526f0d6b5b1b74bc779de97a269c64931ca: Status 404 returned error can't find the container with id a3a960ffd25a6397e364926562763526f0d6b5b1b74bc779de97a269c64931ca
Jul 22 01:28:03.196673 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.196654    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xclt9"
Jul 22 01:28:03.201391 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.201372    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:03.204152 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.204129    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe14439e_c018_434a_b8f7_cf84431ae73b.slice/crio-896f800d3bc5a63e1107647d503b23065e957e8e329752c579853c9b7df987b1 WatchSource:0}: Error finding container 896f800d3bc5a63e1107647d503b23065e957e8e329752c579853c9b7df987b1: Status 404 returned error can't find the container with id 896f800d3bc5a63e1107647d503b23065e957e8e329752c579853c9b7df987b1
Jul 22 01:28:03.212024 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.212003    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3b75e8_d9b2_42d6_8947_70ea7629fea8.slice/crio-9fd33bfd0b2b385d1b3a90e3d417c1bbbdfae69e451c1a48b6ed299b58ac3c53 WatchSource:0}: Error finding container 9fd33bfd0b2b385d1b3a90e3d417c1bbbdfae69e451c1a48b6ed299b58ac3c53: Status 404 returned error can't find the container with id 9fd33bfd0b2b385d1b3a90e3d417c1bbbdfae69e451c1a48b6ed299b58ac3c53
Jul 22 01:28:03.242700 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.242668    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xx4b8"
Jul 22 01:28:03.252235 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.252208    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1545bd2f_f722_4fef_9083_4977fed190d7.slice/crio-1d893ec0a6ef93137d2314f44c6bded1eb2c9dc159d64140fa8f736433d1905b WatchSource:0}: Error finding container 1d893ec0a6ef93137d2314f44c6bded1eb2c9dc159d64140fa8f736433d1905b: Status 404 returned error can't find the container with id 1d893ec0a6ef93137d2314f44c6bded1eb2c9dc159d64140fa8f736433d1905b
Jul 22 01:28:03.255417 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.255361    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg"
Jul 22 01:28:03.263289 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.263268    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e491c8_6fc3_4102_aa46_99f673c17ae0.slice/crio-492fa061241217acab191b9e9f8f21f6dc2c943410d988a6ca507a4586079ff6 WatchSource:0}: Error finding container 492fa061241217acab191b9e9f8f21f6dc2c943410d988a6ca507a4586079ff6: Status 404 returned error can't find the container with id 492fa061241217acab191b9e9f8f21f6dc2c943410d988a6ca507a4586079ff6
Jul 22 01:28:03.273856 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.273837    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:03.275572 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.275555    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-b88qd"
Jul 22 01:28:03.280360 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.280338    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngqhh"
Jul 22 01:28:03.284209 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.284191    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43001b2_2152_4107_8421_cf2cab3e7e7b.slice/crio-895d904b949a5902ad6572a226fdf96ba2370bdcc9aca6ef231b73af69cd8ccc WatchSource:0}: Error finding container 895d904b949a5902ad6572a226fdf96ba2370bdcc9aca6ef231b73af69cd8ccc: Status 404 returned error can't find the container with id 895d904b949a5902ad6572a226fdf96ba2370bdcc9aca6ef231b73af69cd8ccc
Jul 22 01:28:03.289098 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:03.289075    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ef6ede_818c_4d00_a20a_6d197988f37c.slice/crio-7f8969e4a24a1b6f79d20274c134dc14f68b79aa0d71b96b5f1c73ae3f0cab23 WatchSource:0}: Error finding container 7f8969e4a24a1b6f79d20274c134dc14f68b79aa0d71b96b5f1c73ae3f0cab23: Status 404 returned error can't find the container with id 7f8969e4a24a1b6f79d20274c134dc14f68b79aa0d71b96b5f1c73ae3f0cab23
Jul 22 01:28:03.515832 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.515803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:03.516015 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.515869    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.516015 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.515965    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:03.516130 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.516043    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:04.516025617 +0000 UTC m=+3.064443718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:03.516130 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.515965    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:03.516130 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.516091    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:04.516076982 +0000 UTC m=+3.064495083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:03.579492 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.579461    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:03.617114 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.617080    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:03.617289 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.617269    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:03.617349 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.617299    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:03.617349 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.617312    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:03.617537 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.617374    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:04.617354891 +0000 UTC m=+3.165773008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:03.841723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.841627    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 01:23:02 +0000 UTC" deadline="2027-12-23 02:04:32.851444464 +0000 UTC"
Jul 22 01:28:03.841723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.841679    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12456h36m29.009772706s"
Jul 22 01:28:03.987373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.987343    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:03.987553 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.987475    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:03.988061 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:03.987876    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:03.988061 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:03.987967    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:04.012806 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.012749    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b88qd" event={"ID":"c43001b2-2152-4107-8421-cf2cab3e7e7b","Type":"ContainerStarted","Data":"895d904b949a5902ad6572a226fdf96ba2370bdcc9aca6ef231b73af69cd8ccc"}
Jul 22 01:28:04.023752 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.022761    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerStarted","Data":"1d893ec0a6ef93137d2314f44c6bded1eb2c9dc159d64140fa8f736433d1905b"}
Jul 22 01:28:04.031120 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.031062    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g2v4g" event={"ID":"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf","Type":"ContainerStarted","Data":"a3a960ffd25a6397e364926562763526f0d6b5b1b74bc779de97a269c64931ca"}
Jul 22 01:28:04.046674 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.046591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal" event={"ID":"30e58754db9f26f88af6667bbe43cbd9","Type":"ContainerStarted","Data":"7c25fa59ea4d0362614a55d63fc4bbb818364bd9992cf1d01645758b5ceda4f5"}
Jul 22 01:28:04.063030 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.062980    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg" event={"ID":"f2e491c8-6fc3-4102-aa46-99f673c17ae0","Type":"ContainerStarted","Data":"492fa061241217acab191b9e9f8f21f6dc2c943410d988a6ca507a4586079ff6"}
Jul 22 01:28:04.072440 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.072404    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lh9bc" event={"ID":"fc3b75e8-d9b2-42d6-8947-70ea7629fea8","Type":"ContainerStarted","Data":"9fd33bfd0b2b385d1b3a90e3d417c1bbbdfae69e451c1a48b6ed299b58ac3c53"}
Jul 22 01:28:04.085182 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.085150    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xclt9" event={"ID":"fe14439e-c018-434a-b8f7-cf84431ae73b","Type":"ContainerStarted","Data":"896f800d3bc5a63e1107647d503b23065e957e8e329752c579853c9b7df987b1"}
Jul 22 01:28:04.107032 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.107001    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"40a65d9d5049faab5ccc50cd0bded0c2b50e0d310999646086c7c91828be1e2c"}
Jul 22 01:28:04.121792 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.121757    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tx4l4" event={"ID":"97c8d377-9f37-4601-b111-36d80763e116","Type":"ContainerStarted","Data":"26a93bb555d165c3acaaaffc2b66c266573c4051e1a71600aceb8b123415ca34"}
Jul 22 01:28:04.146486 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.146431    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal" event={"ID":"3a5bc24e7d34684078d3262a35f51e63","Type":"ContainerStarted","Data":"961c28257b1a3725a9b2cb549059556450d36866099f4f934b0bf42830a24352"}
Jul 22 01:28:04.159721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.159686    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngqhh" event={"ID":"c6ef6ede-818c-4d00-a20a-6d197988f37c","Type":"ContainerStarted","Data":"7f8969e4a24a1b6f79d20274c134dc14f68b79aa0d71b96b5f1c73ae3f0cab23"}
Jul 22 01:28:04.394699 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.394622    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:04.527277 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.527241    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:04.527553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.527308    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:04.527553 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.527381    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:04.527553 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.527420    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:04.527553 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.527456    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:06.52743832 +0000 UTC m=+5.075856437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:04.527553 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.527477    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:06.527466788 +0000 UTC m=+5.075884902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:04.628265 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.628228    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:04.628472 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.628441    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:04.628472 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.628465    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:04.628585 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.628478    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:04.628585 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.628541    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:06.62852367 +0000 UTC m=+5.176941791 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:04.843093 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.842985    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 01:23:02 +0000 UTC" deadline="2027-12-26 18:37:52.795901908 +0000 UTC"
Jul 22 01:28:04.843093 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.843047    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12545h9m47.952859448s"
Jul 22 01:28:04.985473 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:04.985438    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:04.985630 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:04.985582    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:05.316672 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:05.316639    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 01:28:05.985217 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:05.985181    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:05.985664 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:05.985321    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:05.985744 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:05.985725    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:05.985838 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:05.985813    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:06.546456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:06.546366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:06.546456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:06.546448    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:06.546676 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.546579    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:06.546676 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.546643    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:10.546623705 +0000 UTC m=+9.095041810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:06.546786 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.546707    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:06.546786 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.546739    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:10.546729351 +0000 UTC m=+9.095147453 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:06.647503 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:06.647464    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:06.647703 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.647669    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:06.647703 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.647692    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:06.647703 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.647705    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:06.647848 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.647763    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:10.64774495 +0000 UTC m=+9.196163055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:06.985390 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:06.985356    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:06.985963 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:06.985523    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:07.984933 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:07.984890    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:07.985082 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:07.985030    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:07.985159 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:07.985080    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:07.985206 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:07.985151    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:08.984691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:08.984654    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:08.985134 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:08.984824    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:09.985429 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:09.985317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:09.986363 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:09.986002    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:09.986363 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:09.986099    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:09.986363 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:09.986191    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:10.582218 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:10.582175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:10.582421 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:10.582248    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:10.582421 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.582359    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:10.582421 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.582362    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:10.582586 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.582434    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:18.582413942 +0000 UTC m=+17.130832044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:10.582586 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.582452    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:18.582442657 +0000 UTC m=+17.130860759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:10.682647 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:10.682567    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:10.682829 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.682719    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:10.682829 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.682738    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:10.682829 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.682750    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:10.682829 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.682814    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:18.682795425 +0000 UTC m=+17.231213538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:10.985307 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:10.985209    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:10.985510 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:10.985351    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:11.987274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:11.987240    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:11.987722 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:11.987358    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:11.987807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:11.987722    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:11.987859 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:11.987824    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:12.985131 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:12.985097    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:12.985311 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:12.985238    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:13.985215 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:13.985166    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:13.985677 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:13.985307    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:13.985677 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:13.985358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:13.985677 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:13.985500    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:14.985630 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:14.985598    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:14.986024 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:14.985731    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:15.985061 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:15.985023    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:15.985237 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:15.985162    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:15.985237 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:15.985214    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:15.985347 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:15.985303    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:16.984719 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:16.984684    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:16.985174 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:16.984826    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:17.985261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:17.985221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:17.985711 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:17.985361    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:17.985711 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:17.985415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:17.985711 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:17.985517    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:18.645335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:18.645300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:18.645516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:18.645345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:18.645516 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.645469    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:18.645516 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.645479    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:18.645638 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.645526    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:34.645506641 +0000 UTC m=+33.193924761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:18.645638 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.645543    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:28:34.645537213 +0000 UTC m=+33.193955314 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:18.745876 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:18.745847    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:18.746041 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.746019    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:18.746088 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.746044    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:18.746088 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.746055    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:18.746170 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.746110    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:28:34.746097081 +0000 UTC m=+33.294515182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:18.984774 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:18.984693    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:18.984979 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:18.984843    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:19.985310 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:19.985276    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:19.985767 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:19.985317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:19.985767 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:19.985424    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:19.985767 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:19.985512    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:20.985510 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:20.985478    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:20.985902 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:20.985603    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:21.985931 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:21.985899    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:21.986414 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:21.986005    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:21.986414 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:21.986051    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:21.986414 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:21.986161    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:22.985564 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:22.985270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:22.985729 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:22.985645    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:23.212692 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.212661    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngqhh" event={"ID":"c6ef6ede-818c-4d00-a20a-6d197988f37c","Type":"ContainerStarted","Data":"9b51435b9f5f163de79746d692c7723cd54cd732ee672aac9b068c30ab761c9f"}
Jul 22 01:28:23.214167 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.214135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-b88qd" event={"ID":"c43001b2-2152-4107-8421-cf2cab3e7e7b","Type":"ContainerStarted","Data":"9ef8aeec5b61cb6b9e28ea6efebfcfaf09196062b75c4c4a82d85262dee65c95"}
Jul 22 01:28:23.218011 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.217983    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="59c79a16786945ff7bcc626cf8bb4cc3930180691aede1aa9b3b273100a5482f" exitCode=0
Jul 22 01:28:23.218115 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.218071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"59c79a16786945ff7bcc626cf8bb4cc3930180691aede1aa9b3b273100a5482f"}
Jul 22 01:28:23.219631 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.219609    2560 generic.go:356] "Generic (PLEG): container finished" podID="30e58754db9f26f88af6667bbe43cbd9" containerID="bf610e647229065d7787dd12d082281d5c59f12672a4d67ddfa85a3430bfd2cc" exitCode=0
Jul 22 01:28:23.219713 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.219675    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal" event={"ID":"30e58754db9f26f88af6667bbe43cbd9","Type":"ContainerDied","Data":"bf610e647229065d7787dd12d082281d5c59f12672a4d67ddfa85a3430bfd2cc"}
Jul 22 01:28:23.221170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.221146    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg" event={"ID":"f2e491c8-6fc3-4102-aa46-99f673c17ae0","Type":"ContainerStarted","Data":"b689f788cecb9c82a73850b85cbee9aad91e9b737d589c31a8b78c57d01c7f8c"}
Jul 22 01:28:23.222433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.222411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lh9bc" event={"ID":"fc3b75e8-d9b2-42d6-8947-70ea7629fea8","Type":"ContainerStarted","Data":"9cf405d207773e70cbb6733be83492f0cd16a95794a864532de41d12c195a0d8"}
Jul 22 01:28:23.223569 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.223542    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xclt9" event={"ID":"fe14439e-c018-434a-b8f7-cf84431ae73b","Type":"ContainerStarted","Data":"bd6d380f3aac32a43fd96c5561bacbfba5992d8ac6322df6f6da28ce01f65e19"}
Jul 22 01:28:23.225974 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.225953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"b9aa041fd2bfc3bf653d2cf2c9577bac3758128f929ab8f932dc56598cf04254"}
Jul 22 01:28:23.226068 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.226021    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"5f16a7bc098928f69b118d77607d9b9cdf54708840d8fd3a696c45bc7ccebe4d"}
Jul 22 01:28:23.226137 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.226071    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"d7b02d5e214e21902114de3717ca821f6f858cc5e89955ae999340c904030c30"}
Jul 22 01:28:23.226137 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.226119    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"66202cd14b6312fe85dba4cd0b764e9c5d06c008c4b2aae73733c01924ae8f24"}
Jul 22 01:28:23.226249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.226155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"e3822436a9efed838aab14f6a93808a5c1b9bc2c75b4bd46c764085b6d648bde"}
Jul 22 01:28:23.227448 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.227423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tx4l4" event={"ID":"97c8d377-9f37-4601-b111-36d80763e116","Type":"ContainerStarted","Data":"e9b4ab1ba3be9419ec585bee5ee89ca41d6a36ac619cbd9b462addbdfc03a14a"}
Jul 22 01:28:23.228813 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.228780    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal" event={"ID":"3a5bc24e7d34684078d3262a35f51e63","Type":"ContainerStarted","Data":"58fab6b3d073b41acc13cf23d0b6941f707435a8501c2d46792c5b751cad6088"}
Jul 22 01:28:23.228900 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.228805    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ngqhh" podStartSLOduration=2.021913441 podStartE2EDuration="21.228794745s" totalImagesPullingTime="19.206881304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.22866329 +0000 UTC m=+21.777081415" watchObservedRunningTime="2026-07-22 01:28:23.228794745 +0000 UTC m=+21.777212867"
Jul 22 01:28:23.265979 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.265758    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lh9bc" podStartSLOduration=2.00520744 podStartE2EDuration="21.265743115s" totalImagesPullingTime="19.260535675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.243498833 +0000 UTC m=+21.791916955" watchObservedRunningTime="2026-07-22 01:28:23.265743115 +0000 UTC m=+21.814161238"
Jul 22 01:28:23.281622 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.281564    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-b88qd" podStartSLOduration=2.110132546 podStartE2EDuration="21.281544101s" totalImagesPullingTime="19.171411555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.280939063 +0000 UTC m=+21.829357185" watchObservedRunningTime="2026-07-22 01:28:23.281544101 +0000 UTC m=+21.829962224"
Jul 22 01:28:23.334131 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.334079    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xclt9" podStartSLOduration=2.040475775 podStartE2EDuration="21.334059413s" totalImagesPullingTime="19.293583638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.333819488 +0000 UTC m=+21.882237611" watchObservedRunningTime="2026-07-22 01:28:23.334059413 +0000 UTC m=+21.882477536"
Jul 22 01:28:23.334342 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.334312    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tx4l4" podStartSLOduration=2.995540201 podStartE2EDuration="22.334302774s" totalImagesPullingTime="19.338762573s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.316042658 +0000 UTC m=+21.864460780" watchObservedRunningTime="2026-07-22 01:28:23.334302774 +0000 UTC m=+21.882720897"
Jul 22 01:28:23.347669 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.347613    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-155.ec2.internal" podStartSLOduration=21.347593248 podStartE2EDuration="21.347593248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:23.346807389 +0000 UTC m=+21.895225512" watchObservedRunningTime="2026-07-22 01:28:23.347593248 +0000 UTC m=+21.896011370"
Jul 22 01:28:23.530407 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.530364    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:23.583837 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.583806    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:23.903153 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.903110    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:23.985188 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.985160    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:23.985350 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:23.985284    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:23.985350 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:23.985339    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:23.985493 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:23.985446    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:24.161085 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.161003    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.220820 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.220784    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.231716 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.231684    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g2v4g" event={"ID":"3c3780a4-44d1-47ec-801e-8a3a8b7a2cbf","Type":"ContainerStarted","Data":"50603cccf2347016e250d06dcec55b962e54f7135e8517577d16685e9ade07cc"}
Jul 22 01:28:24.233285 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.233261    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal" event={"ID":"30e58754db9f26f88af6667bbe43cbd9","Type":"ContainerStarted","Data":"21d97619e322597ac17e68c3372f26fae10268301c66c6bba7358f99b3879640"}
Jul 22 01:28:24.235898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.235876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"0a20995a9e85a18ef803ba88014bb4febcbc754b61358b3099bfa1db6b8c8b88"}
Jul 22 01:28:24.250724 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.250682    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g2v4g" podStartSLOduration=3.939427079 podStartE2EDuration="23.250670714s" totalImagesPullingTime="19.311243635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:24.250087076 +0000 UTC m=+22.798505199" watchObservedRunningTime="2026-07-22 01:28:24.250670714 +0000 UTC m=+22.799088836"
Jul 22 01:28:24.263784 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.263745    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-155.ec2.internal" podStartSLOduration=22.263734958 podStartE2EDuration="22.263734958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:24.263285396 +0000 UTC m=+22.811703519" watchObservedRunningTime="2026-07-22 01:28:24.263734958 +0000 UTC m=+22.812153081"
Jul 22 01:28:24.280058 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.280033    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.378262 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.378232    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.469412 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.469320    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.590964 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.590932    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.627593 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.627574    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 01:28:24.683432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.683303    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:24.873642 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.873534    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T01:28:24.627588848Z","UUID":"f060b9eb-a08f-449f-943f-dee76c7722c6","Handler":null,"Name":"","Endpoint":""}
Jul 22 01:28:24.875145 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.875121    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 01:28:24.875273 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.875197    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 01:28:24.985133 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:24.985092    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:24.985342 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:24.985242    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:25.002008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.001971    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:25.048512 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.048483    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngqhh_c6ef6ede-818c-4d00-a20a-6d197988f37c/dns-node-resolver/0.log"
Jul 22 01:28:25.129110 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.129035    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:25.241251 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.241215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg" event={"ID":"f2e491c8-6fc3-4102-aa46-99f673c17ae0","Type":"ContainerStarted","Data":"7e40b92ced9e8c2fdb08c5477ad357783b354ccdab834d8c832073b3f299c6e0"}
Jul 22 01:28:25.847862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.847794    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b88qd_c43001b2-2152-4107-8421-cf2cab3e7e7b/node-ca/0.log"
Jul 22 01:28:25.985314 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.985286    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:25.985454 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:25.985410    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:25.985503 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:25.985455    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:25.985878 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:25.985563    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:26.085618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.085580    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:26.086280 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.086257    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:26.246184 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.246151    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg" event={"ID":"f2e491c8-6fc3-4102-aa46-99f673c17ae0","Type":"ContainerStarted","Data":"dd6e45da25c14c3b083085fed4f51fa45c0ca5253ee575a32e6ad3fff71652f5"}
Jul 22 01:28:26.249101 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.249073    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"1cc8fefff14ddebfdfeed41960a7e63a3ca20716c70cb5d2b3dbd4594ca29c3d"}
Jul 22 01:28:26.483468 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.483436    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:26.584452 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.584359    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:26.683790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.683758    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:26.985187 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:26.985158    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:26.985362 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:26.985276    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:27.381866 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:27.381826    2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log" limit=1024
Jul 22 01:28:27.985741 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:27.985479    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:27.985905 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:27.985799    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:27.986084 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:27.985486    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:27.986214 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:27.986187    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:28.258240 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.258147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" event={"ID":"9c97dcd2-f082-47fd-952b-e78a90c6375f","Type":"ContainerStarted","Data":"1cd43a8194d2a2661b5efe5f5b19c629bc104edd9ef64f096df26334a5a38b76"}
Jul 22 01:28:28.258460 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.258438    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:28.258541 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.258529    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:28.275509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.275483    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:28.297012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.296968    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8rntg" podStartSLOduration=4.337301594 podStartE2EDuration="26.296953503s" totalImagesPullingTime="21.959651909s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:26.279560497 +0000 UTC m=+24.827978620" watchObservedRunningTime="2026-07-22 01:28:28.296953503 +0000 UTC m=+26.845371646"
Jul 22 01:28:28.297589 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.297559    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw" podStartSLOduration=7.9638786249999995 podStartE2EDuration="27.29754879s" totalImagesPullingTime="19.333670165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:28.296085469 +0000 UTC m=+26.844503602" watchObservedRunningTime="2026-07-22 01:28:28.29754879 +0000 UTC m=+26.845966912"
Jul 22 01:28:28.985030 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:28.984997    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:28.985616 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:28.985125    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:29.098038 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.098005    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:29.098387 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.098138    2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 22 01:28:29.098687 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.098662    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lh9bc"
Jul 22 01:28:29.260256 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.260170    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:29.274410 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.274373    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:28:29.984955 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.984920    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:29.985095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:29.984920    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:29.985095 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:29.985023    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:29.985586 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:29.985099    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:30.263761 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:30.263725    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="1831718dae33c8dd4e7cda3ba738e0d600bb5b1aca08974ad1991269d399c3af" exitCode=0
Jul 22 01:28:30.263947 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:30.263807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"1831718dae33c8dd4e7cda3ba738e0d600bb5b1aca08974ad1991269d399c3af"}
Jul 22 01:28:30.984687 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:30.984654    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:30.984829 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:30.984760    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:31.986316 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:31.986093    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:31.986706 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:31.986165    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:31.986706 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:31.986333    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:31.986706 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:31.986464    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:32.270875 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:32.270798    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="68f9860065374bdb167f97ac257d7412e34b92752a55d97f8ff793d640ea6465" exitCode=0
Jul 22 01:28:32.270875 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:32.270839    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"68f9860065374bdb167f97ac257d7412e34b92752a55d97f8ff793d640ea6465"}
Jul 22 01:28:32.985562 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:32.985530    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:32.985701 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:32.985640    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:33.985122 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:33.985037    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:33.985557 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:33.985161    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:33.985557 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:33.985221    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:33.985557 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:33.985295    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:34.277218 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.277140    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="1330be426762364bf66161a14195d0cc0ef7f4dc6c70d77b09adc4946112f7e9" exitCode=0
Jul 22 01:28:34.277218 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.277187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"1330be426762364bf66161a14195d0cc0ef7f4dc6c70d77b09adc4946112f7e9"}
Jul 22 01:28:34.669423 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.669374    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:34.669589 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.669447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:34.669589 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.669536    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:34.669656 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.669592    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:34.669656 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.669608    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret podName:a15a6203-7643-4de5-b179-d107b53699f1 nodeName:}" failed. No retries permitted until 2026-07-22 01:29:06.669587183 +0000 UTC m=+65.218005287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret") pod "global-pull-secret-syncer-8t8r6" (UID: "a15a6203-7643-4de5-b179-d107b53699f1") : object "kube-system"/"original-pull-secret" not registered
Jul 22 01:28:34.669716 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.669664    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs podName:6af796ea-73a6-41c9-a711-e128aacb9a60 nodeName:}" failed. No retries permitted until 2026-07-22 01:29:06.669645977 +0000 UTC m=+65.218064079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs") pod "network-metrics-daemon-gmzh4" (UID: "6af796ea-73a6-41c9-a711-e128aacb9a60") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 01:28:34.770205 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.770169    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:34.770356 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.770321    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 01:28:34.770356 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.770342    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 01:28:34.770356 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.770351    2560 projected.go:196] Error preparing data for projected volume kube-api-access-xpplb for pod openshift-network-diagnostics/network-check-target-7jxjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:34.770466 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.770415    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb podName:379e16a1-94cd-4e3a-b8d4-f851b624d65b nodeName:}" failed. No retries permitted until 2026-07-22 01:29:06.770386966 +0000 UTC m=+65.318805067 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xpplb" (UniqueName: "kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb") pod "network-check-target-7jxjz" (UID: "379e16a1-94cd-4e3a-b8d4-f851b624d65b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 01:28:34.985376 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:34.985291    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:34.985795 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:34.985449    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:35.985188 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:35.985155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:35.985363 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:35.985274    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:35.985363 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:35.985326    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:35.985785 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:35.985445    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:36.984752 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:36.984721    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:36.985003 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:36.984853    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:37.984992 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:37.984951    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:37.985468 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:37.985094    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:37.985468 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:37.985135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:37.985468 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:37.985223    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:38.985524 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:38.985492    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:38.985878 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:38.985606    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:39.290696 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:39.290616    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="ff1865fdd1c0488288aacf97fa86c98ad44f55279eba2832e6e0fd1c52e56f3f" exitCode=0
Jul 22 01:28:39.290696 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:39.290664    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"ff1865fdd1c0488288aacf97fa86c98ad44f55279eba2832e6e0fd1c52e56f3f"}
Jul 22 01:28:39.985164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:39.985135    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:39.985328 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:39.985140    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:39.985328 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:39.985236    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:39.985418 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:39.985347    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:40.298153 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:40.298068    2560 generic.go:356] "Generic (PLEG): container finished" podID="1545bd2f-f722-4fef-9083-4977fed190d7" containerID="46eecfd9e6016bddfd0016233e1157a66fe6064573fc76aa60403e0f69c7f36f" exitCode=0
Jul 22 01:28:40.298504 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:40.298150    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerDied","Data":"46eecfd9e6016bddfd0016233e1157a66fe6064573fc76aa60403e0f69c7f36f"}
Jul 22 01:28:40.985600 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:40.985575    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:40.985768 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:40.985673    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:41.302894 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:41.302821    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" event={"ID":"1545bd2f-f722-4fef-9083-4977fed190d7","Type":"ContainerStarted","Data":"9514bd354bf94e2c9df3c0c5ec1356da62ffaaf20a15ba69d65249812f21773f"}
Jul 22 01:28:41.324838 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:41.324793    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xx4b8" podStartSLOduration=6.833833015 podStartE2EDuration="39.324780812s" totalImagesPullingTime="32.490947797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:41.324577871 +0000 UTC m=+39.872995994" watchObservedRunningTime="2026-07-22 01:28:41.324780812 +0000 UTC m=+39.873198931"
Jul 22 01:28:41.985610 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:41.985571    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:41.985776 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:41.985682    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:41.985776 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:41.985733    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:41.985850 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:41.985817    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:42.985552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:42.985515    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:42.985880 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:42.985623    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:43.985634 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:43.985599    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:43.986059 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:43.985724    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:43.986059 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:43.985777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:43.986059 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:43.985872    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:44.985543 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:44.985506    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:44.985705 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:44.985626    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:45.984987 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:45.984947    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:45.985155 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:45.984994    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:45.985155 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:45.985080    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:45.985223 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:45.985173    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:46.984681 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:46.984647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:46.985060 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:46.984757    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:47.985472 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:47.985441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:47.985818 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:47.985550    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:47.985818 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:47.985604    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:47.985818 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:47.985689    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:48.985625 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:48.985590    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:48.986004 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:48.985701    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:49.910281 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.910234    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jxjz"]
Jul 22 01:28:49.910454 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.910357    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:49.910516 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:49.910493    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:49.912548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.912523    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gmzh4"]
Jul 22 01:28:49.912671 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.912606    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:49.912726 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:49.912689    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:49.922296 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.922233    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8t8r6"]
Jul 22 01:28:49.922423 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:49.922317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:49.922474 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:49.922420    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:51.986289 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:51.986259    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:51.986802 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:51.986378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:51.986802 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:51.986427    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gmzh4" podUID="6af796ea-73a6-41c9-a711-e128aacb9a60"
Jul 22 01:28:51.986802 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:51.986451    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:51.986802 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:51.986455    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7jxjz" podUID="379e16a1-94cd-4e3a-b8d4-f851b624d65b"
Jul 22 01:28:51.986802 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:28:51.986558    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8t8r6" podUID="a15a6203-7643-4de5-b179-d107b53699f1"
Jul 22 01:28:53.259055 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.259026    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-155.ec2.internal" event="NodeReady"
Jul 22 01:28:53.259632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.259180    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 01:28:53.293164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.293133    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64c6f57f95-zk9c8"]
Jul 22 01:28:53.294828 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.294813    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.297265 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.297243    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 01:28:53.297265 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.297246    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 01:28:53.297468 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.297349    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 01:28:53.297612 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.297593    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4rgl4\""
Jul 22 01:28:53.305293 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.305265    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-snkdq"]
Jul 22 01:28:53.305716 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.305694    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 01:28:53.307167 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.307142    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.308590 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.308564    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vk6l2"]
Jul 22 01:28:53.309462 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.309440    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 01:28:53.309541 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.309502    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 01:28:53.309541 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.309509    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gdbrc\""
Jul 22 01:28:53.309650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.309636    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 01:28:53.309741 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.309725    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 01:28:53.310418 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.310382    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c6f57f95-zk9c8"]
Jul 22 01:28:53.310518 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.310507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.312650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.312620    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 01:28:53.312742 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.312671    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjdcf\""
Jul 22 01:28:53.312791 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.312741    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 01:28:53.319500 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.319434    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-snkdq"]
Jul 22 01:28:53.322532 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.322511    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vk6l2"]
Jul 22 01:28:53.408082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5870761-98cb-4f9c-b6a0-853e2181e674-data-volume\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.408082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.408292 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/749625cb-3e84-40a2-996d-f0a81a5a4de9-ca-trust-extracted\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408292 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-bound-sa-token\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408292 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48450b9b-d90c-49ec-b8d6-f975e43a0161-tmp-dir\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.408292 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48450b9b-d90c-49ec-b8d6-f975e43a0161-metrics-tls\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408307    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5870761-98cb-4f9c-b6a0-853e2181e674-crio-socket\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408323    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh4hb\" (UniqueName: \"kubernetes.io/projected/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-api-access-wh4hb\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-certificates\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408374    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-tls\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-trusted-ca\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408445 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5870761-98cb-4f9c-b6a0-853e2181e674-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.408643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-image-registry-private-configuration\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408465    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-installation-pull-secrets\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.408643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408479    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48450b9b-d90c-49ec-b8d6-f975e43a0161-config-volume\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.408643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408499    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwt9g\" (UniqueName: \"kubernetes.io/projected/48450b9b-d90c-49ec-b8d6-f975e43a0161-kube-api-access-qwt9g\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.408643 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.408533    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzwl6\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-kube-api-access-kzwl6\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.409987 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.409964    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qc6md"]
Jul 22 01:28:53.411940 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.411926    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.414297 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.414271    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 01:28:53.414423 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.414274    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 01:28:53.414423 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.414363    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m5cb9\""
Jul 22 01:28:53.414519 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.414445    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 01:28:53.421016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.420992    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qc6md"]
Jul 22 01:28:53.509317 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509279    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5870761-98cb-4f9c-b6a0-853e2181e674-data-volume\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509317 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509535 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509343    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/749625cb-3e84-40a2-996d-f0a81a5a4de9-ca-trust-extracted\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509535 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509470    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-bound-sa-token\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509535 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509508    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48450b9b-d90c-49ec-b8d6-f975e43a0161-tmp-dir\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.509657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50cb7235-e418-4f82-83ee-ddcdb4514803-cert\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.509657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509607    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48450b9b-d90c-49ec-b8d6-f975e43a0161-metrics-tls\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.509749 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5870761-98cb-4f9c-b6a0-853e2181e674-crio-socket\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509749 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509697    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f5870761-98cb-4f9c-b6a0-853e2181e674-data-volume\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509749 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509722    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/749625cb-3e84-40a2-996d-f0a81a5a4de9-ca-trust-extracted\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509745    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh4hb\" (UniqueName: \"kubernetes.io/projected/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-api-access-wh4hb\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-certificates\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-tls\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-trusted-ca\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509850    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f5870761-98cb-4f9c-b6a0-853e2181e674-crio-socket\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.509893 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509869    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5870761-98cb-4f9c-b6a0-853e2181e674-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509894    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-image-registry-private-configuration\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-installation-pull-secrets\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509953    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48450b9b-d90c-49ec-b8d6-f975e43a0161-config-volume\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509893    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48450b9b-d90c-49ec-b8d6-f975e43a0161-tmp-dir\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.509976    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwt9g\" (UniqueName: \"kubernetes.io/projected/48450b9b-d90c-49ec-b8d6-f975e43a0161-kube-api-access-qwt9g\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.510037    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z76pq\" (UniqueName: \"kubernetes.io/projected/50cb7235-e418-4f82-83ee-ddcdb4514803-kube-api-access-z76pq\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.510164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.510072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzwl6\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-kube-api-access-kzwl6\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.510542 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.510514    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48450b9b-d90c-49ec-b8d6-f975e43a0161-config-volume\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.510592 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.510554    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-certificates\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.510909 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.510887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/749625cb-3e84-40a2-996d-f0a81a5a4de9-trusted-ca\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.513958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.513938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48450b9b-d90c-49ec-b8d6-f975e43a0161-metrics-tls\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.513958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.513951    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-installation-pull-secrets\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.514078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.513992    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f5870761-98cb-4f9c-b6a0-853e2181e674-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.514078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.514031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/749625cb-3e84-40a2-996d-f0a81a5a4de9-image-registry-private-configuration\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.514078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.514047    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-registry-tls\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.517819 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.517798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwt9g\" (UniqueName: \"kubernetes.io/projected/48450b9b-d90c-49ec-b8d6-f975e43a0161-kube-api-access-qwt9g\") pod \"dns-default-vk6l2\" (UID: \"48450b9b-d90c-49ec-b8d6-f975e43a0161\") " pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.517900 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.517851    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-bound-sa-token\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.518319 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.518299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh4hb\" (UniqueName: \"kubernetes.io/projected/f5870761-98cb-4f9c-b6a0-853e2181e674-kube-api-access-wh4hb\") pod \"insights-runtime-extractor-snkdq\" (UID: \"f5870761-98cb-4f9c-b6a0-853e2181e674\") " pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.518404 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.518379    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzwl6\" (UniqueName: \"kubernetes.io/projected/749625cb-3e84-40a2-996d-f0a81a5a4de9-kube-api-access-kzwl6\") pod \"image-registry-64c6f57f95-zk9c8\" (UID: \"749625cb-3e84-40a2-996d-f0a81a5a4de9\") " pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.606012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.605986    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:53.610603 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.610577    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50cb7235-e418-4f82-83ee-ddcdb4514803-cert\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.610723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.610651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z76pq\" (UniqueName: \"kubernetes.io/projected/50cb7235-e418-4f82-83ee-ddcdb4514803-kube-api-access-z76pq\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.613142 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.613123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50cb7235-e418-4f82-83ee-ddcdb4514803-cert\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.616507 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.616385    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-snkdq"
Jul 22 01:28:53.618844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.618808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z76pq\" (UniqueName: \"kubernetes.io/projected/50cb7235-e418-4f82-83ee-ddcdb4514803-kube-api-access-z76pq\") pod \"ingress-canary-qc6md\" (UID: \"50cb7235-e418-4f82-83ee-ddcdb4514803\") " pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.621972 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.621944    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:28:53.720532 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.720046    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qc6md"
Jul 22 01:28:53.793470 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.793433    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vk6l2"]
Jul 22 01:28:53.796586 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.796484    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-snkdq"]
Jul 22 01:28:53.800597 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:53.800132    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48450b9b_d90c_49ec_b8d6_f975e43a0161.slice/crio-0c21dbd7b264f19fe8cb0d069ff01faa094dd1f9dc661d37146590b1ded84174 WatchSource:0}: Error finding container 0c21dbd7b264f19fe8cb0d069ff01faa094dd1f9dc661d37146590b1ded84174: Status 404 returned error can't find the container with id 0c21dbd7b264f19fe8cb0d069ff01faa094dd1f9dc661d37146590b1ded84174
Jul 22 01:28:53.802701 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:53.801994    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5870761_98cb_4f9c_b6a0_853e2181e674.slice/crio-b9272e41f0d48c4bbba43ef4070cfa5658aeed8e8840a55fea75bc68ef1e06b1 WatchSource:0}: Error finding container b9272e41f0d48c4bbba43ef4070cfa5658aeed8e8840a55fea75bc68ef1e06b1: Status 404 returned error can't find the container with id b9272e41f0d48c4bbba43ef4070cfa5658aeed8e8840a55fea75bc68ef1e06b1
Jul 22 01:28:53.806168 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.806045    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64c6f57f95-zk9c8"]
Jul 22 01:28:53.862028 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.861937    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qc6md"]
Jul 22 01:28:53.865970 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:28:53.865941    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cb7235_e418_4f82_83ee_ddcdb4514803.slice/crio-04dbc7a42fd59d323030495d5e75974412ee4765d1dafaa67bbaa764d3095fd6 WatchSource:0}: Error finding container 04dbc7a42fd59d323030495d5e75974412ee4765d1dafaa67bbaa764d3095fd6: Status 404 returned error can't find the container with id 04dbc7a42fd59d323030495d5e75974412ee4765d1dafaa67bbaa764d3095fd6
Jul 22 01:28:53.985588 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.985546    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:28:53.985760 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.985546    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:28:53.985760 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.985546    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:28:53.988691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.988515    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 01:28:53.988691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.988560    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 01:28:53.988691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.988589    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 01:28:53.988691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.988600    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l47xs\""
Jul 22 01:28:53.988993 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.988845    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 01:28:53.989047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:53.989033    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vpdlh\""
Jul 22 01:28:54.144031 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.143943    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"]
Jul 22 01:28:54.145956 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.145931    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:28:54.148526 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.148509    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 22 01:28:54.148654 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.148599    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6g4q6\""
Jul 22 01:28:54.155600 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.155575    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"]
Jul 22 01:28:54.217717 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.217681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4251218f-e27c-4e53-a94c-924e59e22d79-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pdjhc\" (UID: \"4251218f-e27c-4e53-a94c-924e59e22d79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:28:54.318439 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.318386    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4251218f-e27c-4e53-a94c-924e59e22d79-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pdjhc\" (UID: \"4251218f-e27c-4e53-a94c-924e59e22d79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:28:54.323439 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.323409    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4251218f-e27c-4e53-a94c-924e59e22d79-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pdjhc\" (UID: \"4251218f-e27c-4e53-a94c-924e59e22d79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:28:54.334012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.333974    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk6l2" event={"ID":"48450b9b-d90c-49ec-b8d6-f975e43a0161","Type":"ContainerStarted","Data":"0c21dbd7b264f19fe8cb0d069ff01faa094dd1f9dc661d37146590b1ded84174"}
Jul 22 01:28:54.335457 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.335430    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qc6md" event={"ID":"50cb7235-e418-4f82-83ee-ddcdb4514803","Type":"ContainerStarted","Data":"04dbc7a42fd59d323030495d5e75974412ee4765d1dafaa67bbaa764d3095fd6"}
Jul 22 01:28:54.336810 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.336785    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snkdq" event={"ID":"f5870761-98cb-4f9c-b6a0-853e2181e674","Type":"ContainerStarted","Data":"b9272e41f0d48c4bbba43ef4070cfa5658aeed8e8840a55fea75bc68ef1e06b1"}
Jul 22 01:28:54.338679 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.338656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8" event={"ID":"749625cb-3e84-40a2-996d-f0a81a5a4de9","Type":"ContainerStarted","Data":"55ddb449feea5af1083ecae73bc80cb44a6271b98f0c98710a06f9f4cc677bb7"}
Jul 22 01:28:54.338768 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.338700    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8" event={"ID":"749625cb-3e84-40a2-996d-f0a81a5a4de9","Type":"ContainerStarted","Data":"bd5adb0c72803ac49ef51590edc2c598cdfaeba8a4902c6efb05b2b568dbe233"}
Jul 22 01:28:54.338908 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.338890    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:28:54.367490 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.367427    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8" podStartSLOduration=6.367387833 podStartE2EDuration="6.367387833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:54.365957984 +0000 UTC m=+52.914376127" watchObservedRunningTime="2026-07-22 01:28:54.367387833 +0000 UTC m=+52.915805954"
Jul 22 01:28:54.454320 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.454233    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:28:54.613935 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:54.613901    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"]
Jul 22 01:28:55.346317 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:55.346246    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc" event={"ID":"4251218f-e27c-4e53-a94c-924e59e22d79","Type":"ContainerStarted","Data":"681bdfe7398b9d05944685d6789d4a86b17c0e3781a0dceb32a6d0bdc3f8a340"}
Jul 22 01:28:56.352927 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:56.352308    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk6l2" event={"ID":"48450b9b-d90c-49ec-b8d6-f975e43a0161","Type":"ContainerStarted","Data":"0346fde292918bec076b27b41ba9aa0e8df2a56df731891c0aaa63ec32984ee2"}
Jul 22 01:28:56.352927 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:56.352435    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk6l2" event={"ID":"48450b9b-d90c-49ec-b8d6-f975e43a0161","Type":"ContainerStarted","Data":"d278c1f3708449c79b6d45fcceb4136a67792b086d371636502ece38f802c145"}
Jul 22 01:28:56.352927 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:28:56.352486    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:29:00.369777 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.369743    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc" event={"ID":"4251218f-e27c-4e53-a94c-924e59e22d79","Type":"ContainerStarted","Data":"a2126f36f3f8e943baaba5d06a1a5303a58ff0d39d9915bd6d351b2a9d2208ee"}
Jul 22 01:29:00.370266 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.369834    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:29:00.371844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.371815    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qc6md" event={"ID":"50cb7235-e418-4f82-83ee-ddcdb4514803","Type":"ContainerStarted","Data":"5528e4f8887dde63d1e15d74451b44a44ffb81432e6c4c1eed8123881956dbc8"}
Jul 22 01:29:00.373181 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.373156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snkdq" event={"ID":"f5870761-98cb-4f9c-b6a0-853e2181e674","Type":"ContainerStarted","Data":"677ef0661f82ccf7a68373c990851135cf577ed4d781a6dddf8ed257c3ed7f8b"}
Jul 22 01:29:00.374642 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.374625    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc"
Jul 22 01:29:00.385384 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.385351    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pdjhc" podStartSLOduration=1.162895424 podStartE2EDuration="6.385341077s" totalImagesPullingTime="5.222445653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:00.385178482 +0000 UTC m=+58.933596605" watchObservedRunningTime="2026-07-22 01:29:00.385341077 +0000 UTC m=+58.933759178"
Jul 22 01:29:00.385623 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.385592    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vk6l2" podStartSLOduration=5.156497391 podStartE2EDuration="7.385584417s" totalImagesPullingTime="2.229087026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:28:56.370626471 +0000 UTC m=+54.919044594" watchObservedRunningTime="2026-07-22 01:29:00.385584417 +0000 UTC m=+58.934002542"
Jul 22 01:29:00.400409 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:00.400360    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qc6md" podStartSLOduration=1.422634578 podStartE2EDuration="7.400348142s" totalImagesPullingTime="5.977713564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:00.399708614 +0000 UTC m=+58.948126736" watchObservedRunningTime="2026-07-22 01:29:00.400348142 +0000 UTC m=+58.948766263"
Jul 22 01:29:01.278410 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:01.278361    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rdrdw"
Jul 22 01:29:04.388144 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:04.388106    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snkdq" event={"ID":"f5870761-98cb-4f9c-b6a0-853e2181e674","Type":"ContainerStarted","Data":"a51f43d2759ec200a1bc5ba237f619f66e0bfc754725fd61cc6975b1cb4fe389"}
Jul 22 01:29:06.357493 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.357458    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vk6l2"
Jul 22 01:29:06.724053 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.723970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:29:06.724053 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.724037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:29:06.726627 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.726600    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 01:29:06.726627 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.726622    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 01:29:06.737555 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.737532    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6af796ea-73a6-41c9-a711-e128aacb9a60-metrics-certs\") pod \"network-metrics-daemon-gmzh4\" (UID: \"6af796ea-73a6-41c9-a711-e128aacb9a60\") " pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:29:06.737687 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.737569    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a15a6203-7643-4de5-b179-d107b53699f1-original-pull-secret\") pod \"global-pull-secret-syncer-8t8r6\" (UID: \"a15a6203-7643-4de5-b179-d107b53699f1\") " pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:29:06.824933 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.824893    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:29:06.827818 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.827782    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 01:29:06.838473 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.838445    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 01:29:06.849207 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.849184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpplb\" (UniqueName: \"kubernetes.io/projected/379e16a1-94cd-4e3a-b8d4-f851b624d65b-kube-api-access-xpplb\") pod \"network-check-target-7jxjz\" (UID: \"379e16a1-94cd-4e3a-b8d4-f851b624d65b\") " pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:29:06.898078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.898047    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8t8r6"
Jul 22 01:29:06.905829 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.905804    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l47xs\""
Jul 22 01:29:06.911079 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.911058    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vpdlh\""
Jul 22 01:29:06.913909 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.913891    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gmzh4"
Jul 22 01:29:06.919527 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:06.919507    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:29:07.513239 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:07.513208    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8t8r6"]
Jul 22 01:29:07.516812 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:07.516786    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15a6203_7643_4de5_b179_d107b53699f1.slice/crio-0cb907d5d5a74e6ebb2551d36a153b4177ddeba666a01bc6df1b4bb463eb77af WatchSource:0}: Error finding container 0cb907d5d5a74e6ebb2551d36a153b4177ddeba666a01bc6df1b4bb463eb77af: Status 404 returned error can't find the container with id 0cb907d5d5a74e6ebb2551d36a153b4177ddeba666a01bc6df1b4bb463eb77af
Jul 22 01:29:07.734653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:07.734570    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7jxjz"]
Jul 22 01:29:07.735336 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:07.735317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gmzh4"]
Jul 22 01:29:07.737563 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:07.737531    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379e16a1_94cd_4e3a_b8d4_f851b624d65b.slice/crio-5dd8a55ff3abe532303b89f694bd6b2d25d8e3f5adacd2594007d428a162f425 WatchSource:0}: Error finding container 5dd8a55ff3abe532303b89f694bd6b2d25d8e3f5adacd2594007d428a162f425: Status 404 returned error can't find the container with id 5dd8a55ff3abe532303b89f694bd6b2d25d8e3f5adacd2594007d428a162f425
Jul 22 01:29:07.738205 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:07.738185    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af796ea_73a6_41c9_a711_e128aacb9a60.slice/crio-cb0953cb3624ecbfd70a2a129ba3a51c6122c27329cb878fea6e0d0094f0cf4c WatchSource:0}: Error finding container cb0953cb3624ecbfd70a2a129ba3a51c6122c27329cb878fea6e0d0094f0cf4c: Status 404 returned error can't find the container with id cb0953cb3624ecbfd70a2a129ba3a51c6122c27329cb878fea6e0d0094f0cf4c
Jul 22 01:29:08.403278 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.403236    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8t8r6" event={"ID":"a15a6203-7643-4de5-b179-d107b53699f1","Type":"ContainerStarted","Data":"0cb907d5d5a74e6ebb2551d36a153b4177ddeba666a01bc6df1b4bb463eb77af"}
Jul 22 01:29:08.405531 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.405495    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-snkdq" event={"ID":"f5870761-98cb-4f9c-b6a0-853e2181e674","Type":"ContainerStarted","Data":"f6f71bf49d9487ee410040496e6d1de73846b87e20992ed8eed377850c79467b"}
Jul 22 01:29:08.406719 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.406692    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gmzh4" event={"ID":"6af796ea-73a6-41c9-a711-e128aacb9a60","Type":"ContainerStarted","Data":"cb0953cb3624ecbfd70a2a129ba3a51c6122c27329cb878fea6e0d0094f0cf4c"}
Jul 22 01:29:08.407796 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.407773    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jxjz" event={"ID":"379e16a1-94cd-4e3a-b8d4-f851b624d65b","Type":"ContainerStarted","Data":"5dd8a55ff3abe532303b89f694bd6b2d25d8e3f5adacd2594007d428a162f425"}
Jul 22 01:29:08.424306 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.424258    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-snkdq" podStartSLOduration=1.907062846 podStartE2EDuration="15.424243553s" totalImagesPullingTime="13.517180707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:08.423161665 +0000 UTC m=+66.971579814" watchObservedRunningTime="2026-07-22 01:29:08.424243553 +0000 UTC m=+66.972661676"
Jul 22 01:29:08.552176 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.552139    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"]
Jul 22 01:29:08.556638 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.556612    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.559723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559538    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 22 01:29:08.559723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559593    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hfbr9\""
Jul 22 01:29:08.559723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559660    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 22 01:29:08.560068 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559812    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 01:29:08.560068 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559866    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 01:29:08.560068 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.559916    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 01:29:08.568802 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.568731    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"]
Jul 22 01:29:08.590954 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.590923    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-26d4w"]
Jul 22 01:29:08.594136 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.594116    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.596317 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.596289    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 01:29:08.596642 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.596627    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 01:29:08.596721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.596709    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fvrqr\""
Jul 22 01:29:08.596850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.596823    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 01:29:08.642199 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.642167    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.642389 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.642215    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.642389 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.642285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfzp6\" (UniqueName: \"kubernetes.io/projected/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-kube-api-access-hfzp6\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.642389 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.642325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.743438 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743341    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-wtmp\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743438 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743388    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-metrics-client-ca\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfzp6\" (UniqueName: \"kubernetes.io/projected/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-kube-api-access-hfzp6\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743517    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743583    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-textfile\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743608    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743636    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.743917 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743668    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.743917 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743917 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-root\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.743917 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-sys\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.744043 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.743917    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4k8m\" (UniqueName: \"kubernetes.io/projected/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-kube-api-access-n4k8m\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.744455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.744425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.746492 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.746442    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.746603 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.746577    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.751379 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.751359    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfzp6\" (UniqueName: \"kubernetes.io/projected/d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b-kube-api-access-hfzp6\") pod \"openshift-state-metrics-79f6958cdf-6pxfs\" (UID: \"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:08.845362 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-textfile\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845423    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845451    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-root\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845732 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845542    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-sys\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845732 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845573    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4k8m\" (UniqueName: \"kubernetes.io/projected/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-kube-api-access-n4k8m\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845732 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-wtmp\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.845732 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845619    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-metrics-client-ca\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-accelerators-collector-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.845970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-root\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.846102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-metrics-client-ca\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.846167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-sys\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.846206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-wtmp\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:29:08.846281    2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:29:08.846336    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls podName:e1518b7f-525f-4bc5-b72b-3e83d66e5fd3 nodeName:}" failed. No retries permitted until 2026-07-22 01:29:09.346316572 +0000 UTC m=+67.894734680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls") pod "node-exporter-26d4w" (UID: "e1518b7f-525f-4bc5-b72b-3e83d66e5fd3") : secret "node-exporter-tls" not found
Jul 22 01:29:08.846578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.846387    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-textfile\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.848608 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.848566    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.855077 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.855056    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4k8m\" (UniqueName: \"kubernetes.io/projected/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-kube-api-access-n4k8m\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:08.867721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:08.867699    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"
Jul 22 01:29:09.029048 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.029010    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs"]
Jul 22 01:29:09.033986 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:09.033955    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d61fe2_4555_4c6d_b971_d4fd9ae3d95b.slice/crio-53df7515bf770cbed223e2cd443bb6ce6051e9892c21746e58ca72e8617eb1f7 WatchSource:0}: Error finding container 53df7515bf770cbed223e2cd443bb6ce6051e9892c21746e58ca72e8617eb1f7: Status 404 returned error can't find the container with id 53df7515bf770cbed223e2cd443bb6ce6051e9892c21746e58ca72e8617eb1f7
Jul 22 01:29:09.354376 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.354285    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:09.356961 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.356931    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e1518b7f-525f-4bc5-b72b-3e83d66e5fd3-node-exporter-tls\") pod \"node-exporter-26d4w\" (UID: \"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3\") " pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:09.414588 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.414174    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gmzh4" event={"ID":"6af796ea-73a6-41c9-a711-e128aacb9a60","Type":"ContainerStarted","Data":"fc094e562c723be4917dfebdf28f04ee43e04d902b0cd2e7d66e2eca350f2807"}
Jul 22 01:29:09.414588 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.414278    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gmzh4" event={"ID":"6af796ea-73a6-41c9-a711-e128aacb9a60","Type":"ContainerStarted","Data":"46aa6de96fe453c37039976ddd461cc75f943803b84cc2dee48282532a887a11"}
Jul 22 01:29:09.416760 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.416725    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs" event={"ID":"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b","Type":"ContainerStarted","Data":"ec3d076eff520cb655de76d34ca68098c039f6d1cbf9703ec75c9d175c42f63d"}
Jul 22 01:29:09.416849 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.416790    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs" event={"ID":"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b","Type":"ContainerStarted","Data":"6b7a0e49463d93c32c2a8aa337a140f55fadfaed54e0cfbaeeeac7975c1b9827"}
Jul 22 01:29:09.416849 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.416819    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs" event={"ID":"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b","Type":"ContainerStarted","Data":"53df7515bf770cbed223e2cd443bb6ce6051e9892c21746e58ca72e8617eb1f7"}
Jul 22 01:29:09.430665 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.430602    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gmzh4" podStartSLOduration=67.382967441 podStartE2EDuration="1m8.430581975s" totalImagesPullingTime="1.047614534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:09.429336831 +0000 UTC m=+67.977754967" watchObservedRunningTime="2026-07-22 01:29:09.430581975 +0000 UTC m=+67.979000099"
Jul 22 01:29:09.505188 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.505155    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26d4w"
Jul 22 01:29:09.532439 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:09.532353    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1518b7f_525f_4bc5_b72b_3e83d66e5fd3.slice/crio-b27398d228d0825b6506f96a3151a6d95423cd6af22f5bea5116c702bb31dac6 WatchSource:0}: Error finding container b27398d228d0825b6506f96a3151a6d95423cd6af22f5bea5116c702bb31dac6: Status 404 returned error can't find the container with id b27398d228d0825b6506f96a3151a6d95423cd6af22f5bea5116c702bb31dac6
Jul 22 01:29:09.634216 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.634187    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:29:09.664728 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.664698    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:29:09.664877 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.664862    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.667878 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.667845    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 22 01:29:09.668011 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.667903    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 22 01:29:09.668011 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.667862    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 22 01:29:09.668122 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.667857    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 22 01:29:09.668122 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.668071    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 22 01:29:09.668214 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.667906    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 22 01:29:09.668214 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.668131    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 22 01:29:09.668375 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.668356    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 22 01:29:09.668471 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.668391    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-p2rsp\""
Jul 22 01:29:09.668668 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.668640    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 22 01:29:09.757691 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.757847 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.757847 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757729    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.757847 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757877    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn8mh\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757953    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.757986    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758198 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.758025    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758198 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.758073    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758198 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.758124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758198 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.758149    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.758377 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.758207    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859155    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859196    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859232    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859259    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859306    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859506    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn8mh\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859628    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859921 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859691    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.859972 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.859921    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.860772 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.860443    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.861658 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.861633    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.863856 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.863160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.864715 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.864691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.864813 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.864743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.864813 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.864761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.864934 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.864910    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.864986 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.864969    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.865260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.865222    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.865260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.865247    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.866068 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.866050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.870855 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.870834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn8mh\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh\") pod \"alertmanager-main-0\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:09.975872 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:09.975762    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:29:10.150884 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:10.150844    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:29:10.423148 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:10.423055    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26d4w" event={"ID":"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3","Type":"ContainerStarted","Data":"b27398d228d0825b6506f96a3151a6d95423cd6af22f5bea5116c702bb31dac6"}
Jul 22 01:29:11.122221 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:11.122192    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e9b3a2_80d7_4a8d_b807_5aef2a1afea4.slice/crio-d59d3ebab0544c0c6a3d349e71011293d49b617e34f4f21ad682837ba70836f8 WatchSource:0}: Error finding container d59d3ebab0544c0c6a3d349e71011293d49b617e34f4f21ad682837ba70836f8: Status 404 returned error can't find the container with id d59d3ebab0544c0c6a3d349e71011293d49b617e34f4f21ad682837ba70836f8
Jul 22 01:29:11.427911 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:11.427834    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"d59d3ebab0544c0c6a3d349e71011293d49b617e34f4f21ad682837ba70836f8"}
Jul 22 01:29:12.960441 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.960390    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"]
Jul 22 01:29:12.962705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.962681    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:12.965229 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.965205    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 22 01:29:12.966260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.965558    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 01:29:12.966260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.965931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x8s8j\""
Jul 22 01:29:12.966260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.966065    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 22 01:29:12.966260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.966132    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 22 01:29:12.966260 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.966193    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7fej0t9k8qniv\""
Jul 22 01:29:12.975242 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:12.975189    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"]
Jul 22 01:29:13.089390 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-client-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089390 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089373    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-tls\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089421    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/721555e7-f881-4c94-834c-9ace160767b4-audit-log\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-metrics-server-audit-profiles\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089480    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-client-certs\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.089594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.089551    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s4tf\" (UniqueName: \"kubernetes.io/projected/721555e7-f881-4c94-834c-9ace160767b4-kube-api-access-4s4tf\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190692 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190657    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/721555e7-f881-4c94-834c-9ace160767b4-audit-log\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-metrics-server-audit-profiles\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190731    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190779    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-client-certs\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190820    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s4tf\" (UniqueName: \"kubernetes.io/projected/721555e7-f881-4c94-834c-9ace160767b4-kube-api-access-4s4tf\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.190868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-client-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.191095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.190896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-tls\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.191498 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.191156    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/721555e7-f881-4c94-834c-9ace160767b4-audit-log\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.191776 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.191752    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.191908 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.191886    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/721555e7-f881-4c94-834c-9ace160767b4-metrics-server-audit-profiles\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.193686 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.193666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-tls\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.193875 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.193853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-secret-metrics-server-client-certs\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.193921 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.193865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/721555e7-f881-4c94-834c-9ace160767b4-client-ca-bundle\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.199262 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.199241    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s4tf\" (UniqueName: \"kubernetes.io/projected/721555e7-f881-4c94-834c-9ace160767b4-kube-api-access-4s4tf\") pod \"metrics-server-56bd4ffb4c-rtqjf\" (UID: \"721555e7-f881-4c94-834c-9ace160767b4\") " pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.276011 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.275983    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:13.331362 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.330905    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"]
Jul 22 01:29:13.334609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.334587    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:13.336572 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.336540    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 22 01:29:13.336807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.336794    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-t5q7s\""
Jul 22 01:29:13.343232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.343120    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"]
Jul 22 01:29:13.440310 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.440272    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs" event={"ID":"d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b","Type":"ContainerStarted","Data":"9c573f0c92b9c7ad4245ba023e8225b8ccac56aa85dda40bbc0a46cd928a16fa"}
Jul 22 01:29:13.442060 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.441989    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26d4w" event={"ID":"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3","Type":"ContainerStarted","Data":"932e3cd80e7f3a785513c8e53fe60ba5617a1bc8f59a374f4e8abc754033df4d"}
Jul 22 01:29:13.443624 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.443559    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7jxjz" event={"ID":"379e16a1-94cd-4e3a-b8d4-f851b624d65b","Type":"ContainerStarted","Data":"2c98b0587a16d430f035872465aba94fbfd8502a8f97c7365f68aeb5106612d1"}
Jul 22 01:29:13.443624 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.443617    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:29:13.445074 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.445051    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8t8r6" event={"ID":"a15a6203-7643-4de5-b179-d107b53699f1","Type":"ContainerStarted","Data":"452b5a543a2b6207642ef3745ce083e6d1fd96fb576c6e88edf6dc08f8eaed35"}
Jul 22 01:29:13.446699 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.446673    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6" exitCode=0
Jul 22 01:29:13.446794 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.446745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"}
Jul 22 01:29:13.459637 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.459585    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-6pxfs" podStartSLOduration=1.9077789250000001 podStartE2EDuration="5.459563493s" totalImagesPullingTime="3.551784568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:13.458139037 +0000 UTC m=+72.006557160" watchObservedRunningTime="2026-07-22 01:29:13.459563493 +0000 UTC m=+72.007981617"
Jul 22 01:29:13.468812 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.468781    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"]
Jul 22 01:29:13.471740 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:13.471716    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721555e7_f881_4c94_834c_9ace160767b4.slice/crio-66df67422504627a6e4e13c371e0a732e5137080163c23a0f663755017e0d926 WatchSource:0}: Error finding container 66df67422504627a6e4e13c371e0a732e5137080163c23a0f663755017e0d926: Status 404 returned error can't find the container with id 66df67422504627a6e4e13c371e0a732e5137080163c23a0f663755017e0d926
Jul 22 01:29:13.473610 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.473569    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7jxjz" podStartSLOduration=66.472347714 podStartE2EDuration="1m11.473550288s" totalImagesPullingTime="5.001202574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:13.473166293 +0000 UTC m=+72.021584418" watchObservedRunningTime="2026-07-22 01:29:13.473550288 +0000 UTC m=+72.021968413"
Jul 22 01:29:13.489716 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.489674    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8t8r6" podStartSLOduration=66.241738714 podStartE2EDuration="1m11.489659822s" totalImagesPullingTime="5.247921108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 01:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:13.48900223 +0000 UTC m=+72.037420353" watchObservedRunningTime="2026-07-22 01:29:13.489659822 +0000 UTC m=+72.038077945"
Jul 22 01:29:13.494218 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.494198    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:13.494310 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.494231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgntl\" (UniqueName: \"kubernetes.io/projected/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-kube-api-access-fgntl\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:13.594896 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.594807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:13.595050 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:29:13.594955    2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 22 01:29:13.595050 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:29:13.595024    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert podName:f4cb14e4-128d-483f-baac-4b4beb8c8bb3 nodeName:}" failed. No retries permitted until 2026-07-22 01:29:14.0950008 +0000 UTC m=+72.643418917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-jwpm5" (UID: "f4cb14e4-128d-483f-baac-4b4beb8c8bb3") : secret "monitoring-plugin-cert" not found
Jul 22 01:29:13.595170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.595074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgntl\" (UniqueName: \"kubernetes.io/projected/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-kube-api-access-fgntl\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:13.608786 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:13.608758    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgntl\" (UniqueName: \"kubernetes.io/projected/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-kube-api-access-fgntl\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:14.099723 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.099689    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:14.101999 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.101981    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4cb14e4-128d-483f-baac-4b4beb8c8bb3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-jwpm5\" (UID: \"f4cb14e4-128d-483f-baac-4b4beb8c8bb3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:14.245961 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.245932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:14.405538 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.405388    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"]
Jul 22 01:29:14.408656 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:14.408624    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4cb14e4_128d_483f_baac_4b4beb8c8bb3.slice/crio-af7814065a2c973cc496b84bc51b96aa4eface5e64fe2953084dbc702501265c WatchSource:0}: Error finding container af7814065a2c973cc496b84bc51b96aa4eface5e64fe2953084dbc702501265c: Status 404 returned error can't find the container with id af7814065a2c973cc496b84bc51b96aa4eface5e64fe2953084dbc702501265c
Jul 22 01:29:14.452573 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.452534    2560 generic.go:356] "Generic (PLEG): container finished" podID="e1518b7f-525f-4bc5-b72b-3e83d66e5fd3" containerID="932e3cd80e7f3a785513c8e53fe60ba5617a1bc8f59a374f4e8abc754033df4d" exitCode=0
Jul 22 01:29:14.452725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.452622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26d4w" event={"ID":"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3","Type":"ContainerDied","Data":"932e3cd80e7f3a785513c8e53fe60ba5617a1bc8f59a374f4e8abc754033df4d"}
Jul 22 01:29:14.453925 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.453900    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf" event={"ID":"721555e7-f881-4c94-834c-9ace160767b4","Type":"ContainerStarted","Data":"66df67422504627a6e4e13c371e0a732e5137080163c23a0f663755017e0d926"}
Jul 22 01:29:14.455375 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.455317    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5" event={"ID":"f4cb14e4-128d-483f-baac-4b4beb8c8bb3","Type":"ContainerStarted","Data":"af7814065a2c973cc496b84bc51b96aa4eface5e64fe2953084dbc702501265c"}
Jul 22 01:29:14.756493 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.756380    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:29:14.760448 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.760426    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.764838 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.764814    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 01:29:14.765002 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.764813    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 01:29:14.765002 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.764813    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 01:29:14.765249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765213    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 01:29:14.765336 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 01:29:14.765336 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765313    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 01:29:14.765336 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765319    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cclvl\""
Jul 22 01:29:14.765336 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765320    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 01:29:14.765537 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 01:29:14.765573 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765546    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5ap6ao3rpn71n\""
Jul 22 01:29:14.765741 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765728    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 01:29:14.765964 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.765950    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 01:29:14.766558 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.766545    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 01:29:14.766636 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.766623    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 01:29:14.768728 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.768712    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 01:29:14.778362 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.778334    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 01:29:14.779987 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.779966    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:29:14.906120 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906085    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906222    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906276    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906298    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906367    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw4ds\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906448    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906506    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906549    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906942 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906641    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906942 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906942 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906942 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906793    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:14.906942 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:14.906855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.007811 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.007811 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007769    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw4ds\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007796    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007944    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008016 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.007994    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008047    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008112    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008172    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008275    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008299    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008853 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008683    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.008853 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.008842    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.009712 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.009382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.010478 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.010025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.011459 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.011431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.012062 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.012040    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.012165 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.012069    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.012165 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.012109    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.012546 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.012526    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.013151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.013123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.013750 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.013726    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.013962 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.013920    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.014979 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.014948    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.015932 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.015477    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.015932 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.015504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.015932 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.015897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.016112 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.015987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.016112 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.016012    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.017457 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.017377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.017889 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.017868    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw4ds\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds\") pod \"prometheus-k8s-0\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.071006 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.070968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:15.351248 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.351178    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64c6f57f95-zk9c8"
Jul 22 01:29:15.462620 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.462584    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26d4w" event={"ID":"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3","Type":"ContainerStarted","Data":"5d18ad03dbc6a62a9fc4e45ea3c3a1da40b6c0cb5010b12de3f5a68752269622"}
Jul 22 01:29:15.462800 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.462644    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26d4w" event={"ID":"e1518b7f-525f-4bc5-b72b-3e83d66e5fd3","Type":"ContainerStarted","Data":"289fb98fe49f7a803e9c84b7e6eaec10b0e36eba2d1ae21ba945a19473ed01fc"}
Jul 22 01:29:15.482523 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.482429    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-26d4w" podStartSLOduration=3.618884259 podStartE2EDuration="7.482407775s" totalImagesPullingTime="3.789936047s" totalInitContainerRuntime="73.587469ms" isStatefulPod=true podCreationTimestamp="2026-07-22 01:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:15.481548335 +0000 UTC m=+74.029966514" watchObservedRunningTime="2026-07-22 01:29:15.482407775 +0000 UTC m=+74.030825892"
Jul 22 01:29:15.849105 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:15.849079    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:29:15.854476 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:29:15.854445    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490c5963_0bc2_4de9_a509_cbef7fb328d5.slice/crio-ea285b72c8cbf36ccf929ed7feb8f0837915c980d6d33a830f66fdc6e7d80152 WatchSource:0}: Error finding container ea285b72c8cbf36ccf929ed7feb8f0837915c980d6d33a830f66fdc6e7d80152: Status 404 returned error can't find the container with id ea285b72c8cbf36ccf929ed7feb8f0837915c980d6d33a830f66fdc6e7d80152
Jul 22 01:29:16.469648 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.469610    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"}
Jul 22 01:29:16.470033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.469703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"}
Jul 22 01:29:16.470033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.469735    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"}
Jul 22 01:29:16.470033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.469766    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"}
Jul 22 01:29:16.470033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.469789    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"}
Jul 22 01:29:16.471051 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.471018    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5" event={"ID":"f4cb14e4-128d-483f-baac-4b4beb8c8bb3","Type":"ContainerStarted","Data":"7d34c0306abee381105654a3d59473231e56b3f2d309a6a084b8f6750e72c6a5"}
Jul 22 01:29:16.471149 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.471067    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:16.472605 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.472578    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf" event={"ID":"721555e7-f881-4c94-834c-9ace160767b4","Type":"ContainerStarted","Data":"27b6e4d15abb6ba4f940700371891844796230e6a14bd12611e184e7219e9008"}
Jul 22 01:29:16.474061 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.474040    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122" exitCode=0
Jul 22 01:29:16.474149 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.474125    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"}
Jul 22 01:29:16.474210 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.474189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"ea285b72c8cbf36ccf929ed7feb8f0837915c980d6d33a830f66fdc6e7d80152"}
Jul 22 01:29:16.476730 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.476709    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5"
Jul 22 01:29:16.490227 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.490178    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-jwpm5" podStartSLOduration=1.519879937 podStartE2EDuration="3.490152762s" totalImagesPullingTime="1.970272825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:16.490072296 +0000 UTC m=+75.038490440" watchObservedRunningTime="2026-07-22 01:29:16.490152762 +0000 UTC m=+75.038570906"
Jul 22 01:29:16.512683 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:16.512637    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf" podStartSLOduration=2.300138728 podStartE2EDuration="4.512625573s" totalImagesPullingTime="2.212486845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:16.511597054 +0000 UTC m=+75.060015177" watchObservedRunningTime="2026-07-22 01:29:16.512625573 +0000 UTC m=+75.061043696"
Jul 22 01:29:18.486427 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:18.486372    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerStarted","Data":"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"}
Jul 22 01:29:18.520615 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:18.520563    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.942072812 podStartE2EDuration="9.520546804s" totalImagesPullingTime="5.561624815s" totalInitContainerRuntime="16.849177ms" isStatefulPod=false podCreationTimestamp="2026-07-22 01:29:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:18.517699798 +0000 UTC m=+77.066117938" watchObservedRunningTime="2026-07-22 01:29:18.520546804 +0000 UTC m=+77.068964951"
Jul 22 01:29:20.495913 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:20.495876    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"}
Jul 22 01:29:20.496268 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:20.495942    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"}
Jul 22 01:29:22.510597 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:22.510501    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"}
Jul 22 01:29:22.510597 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:22.510591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"}
Jul 22 01:29:22.510951 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:22.510628    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"}
Jul 22 01:29:22.510951 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:22.510667    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerStarted","Data":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"}
Jul 22 01:29:22.541818 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:22.541768    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.9856358160000003 podStartE2EDuration="8.5417512s" totalImagesPullingTime="5.530126235s" totalInitContainerRuntime="25.989149ms" isStatefulPod=false podCreationTimestamp="2026-07-22 01:29:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:29:22.539154332 +0000 UTC m=+81.087572455" watchObservedRunningTime="2026-07-22 01:29:22.5417512 +0000 UTC m=+81.090169322"
Jul 22 01:29:25.071367 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:25.071250    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:29:33.277059 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:33.277025    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:33.277544 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:33.277086    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:44.458315 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:44.458278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7jxjz"
Jul 22 01:29:53.282048 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:53.282011    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:29:53.285861 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:29:53.285835    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56bd4ffb4c-rtqjf"
Jul 22 01:30:15.071316 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:15.071278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:15.089945 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:15.089922    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:15.719332 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:15.719305    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:28.900965 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.900929    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:30:28.901432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901236    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="alertmanager" containerID="cri-o://3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7" gracePeriod=120
Jul 22 01:30:28.901432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901302    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-metric" containerID="cri-o://9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6" gracePeriod=120
Jul 22 01:30:28.901432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901348    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-web" containerID="cri-o://562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8" gracePeriod=120
Jul 22 01:30:28.901614 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901435    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="prom-label-proxy" containerID="cri-o://7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac" gracePeriod=120
Jul 22 01:30:28.901614 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901363    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy" containerID="cri-o://514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4" gracePeriod=120
Jul 22 01:30:28.901614 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:28.901367    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="config-reloader" containerID="cri-o://f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf" gracePeriod=120
Jul 22 01:30:29.759794 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759754    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac" exitCode=0
Jul 22 01:30:29.759794 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759778    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4" exitCode=0
Jul 22 01:30:29.759794 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759786    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf" exitCode=0
Jul 22 01:30:29.759794 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759791    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7" exitCode=0
Jul 22 01:30:29.760045 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"}
Jul 22 01:30:29.760045 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759899    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"}
Jul 22 01:30:29.760045 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"}
Jul 22 01:30:29.760045 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:29.759954    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"}
Jul 22 01:30:30.140323 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.140299    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:30:30.250784 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250749    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.250953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250794    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.250953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250822    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.250953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250854    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.250953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250880    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.250953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250922    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250956    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bn8mh\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.250983    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251018    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251065    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251096    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251124    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251164    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets\") pod \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\" (UID: \"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4\") "
Jul 22 01:30:30.251564 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251212    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:30.251564 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251429    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.251564 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251527    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:30:30.251878 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.251854    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:30.253823 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.253668    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.254084 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254046    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.254278 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254253    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh" (OuterVolumeSpecName: "kube-api-access-bn8mh") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "kube-api-access-bn8mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:30:30.254455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254427    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.254577 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254552    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:30:30.254641 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254573    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume" (OuterVolumeSpecName: "config-volume") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.254938 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.254919    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.255929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.255908    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out" (OuterVolumeSpecName: "config-out") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:30:30.258159 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.258131    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.263963 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.263916    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config" (OuterVolumeSpecName: "web-config") pod "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" (UID: "47e9b3a2-80d7-4a8d-b807-5aef2a1afea4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:30.352257 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352210    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-metrics-client-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352257 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352250    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-alertmanager-main-db\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352257 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352261    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-out\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352257 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352271    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352282    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352292    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-tls-assets\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352320    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-web-config\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352330    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-main-tls\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352338    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-config-volume\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352347    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352356    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-cluster-tls-config\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.352536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.352364    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn8mh\" (UniqueName: \"kubernetes.io/projected/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4-kube-api-access-bn8mh\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:30.766668 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766629    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6" exitCode=0
Jul 22 01:30:30.766668 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766658    2560 generic.go:356] "Generic (PLEG): container finished" podID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerID="562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8" exitCode=0
Jul 22 01:30:30.766844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766707    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"}
Jul 22 01:30:30.766844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766724    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 22 01:30:30.766844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766777    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"}
Jul 22 01:30:30.766844 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766806    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"47e9b3a2-80d7-4a8d-b807-5aef2a1afea4","Type":"ContainerDied","Data":"d59d3ebab0544c0c6a3d349e71011293d49b617e34f4f21ad682837ba70836f8"}
Jul 22 01:30:30.766969 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.766846    2560 scope.go:117] "RemoveContainer" containerID="7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"
Jul 22 01:30:30.774109 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.774092    2560 scope.go:117] "RemoveContainer" containerID="9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"
Jul 22 01:30:30.780582 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.780559    2560 scope.go:117] "RemoveContainer" containerID="514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"
Jul 22 01:30:30.786956 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.786941    2560 scope.go:117] "RemoveContainer" containerID="562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"
Jul 22 01:30:30.792799 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.792784    2560 scope.go:117] "RemoveContainer" containerID="f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"
Jul 22 01:30:30.800855 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.800826    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:30:30.801487 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.801464    2560 scope.go:117] "RemoveContainer" containerID="3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"
Jul 22 01:30:30.805583 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.805564    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 22 01:30:30.810278 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.810261    2560 scope.go:117] "RemoveContainer" containerID="15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"
Jul 22 01:30:30.816309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.816293    2560 scope.go:117] "RemoveContainer" containerID="7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"
Jul 22 01:30:30.816585 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.816561    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac\": container with ID starting with 7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac not found: ID does not exist" containerID="7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"
Jul 22 01:30:30.816653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.816596    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"} err="failed to get container status \"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac\": rpc error: code = NotFound desc = could not find container \"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac\": container with ID starting with 7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac not found: ID does not exist"
Jul 22 01:30:30.816653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.816642    2560 scope.go:117] "RemoveContainer" containerID="9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"
Jul 22 01:30:30.816884 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.816867    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6\": container with ID starting with 9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6 not found: ID does not exist" containerID="9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"
Jul 22 01:30:30.816924 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.816890    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"} err="failed to get container status \"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6\": rpc error: code = NotFound desc = could not find container \"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6\": container with ID starting with 9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6 not found: ID does not exist"
Jul 22 01:30:30.816924 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.816905    2560 scope.go:117] "RemoveContainer" containerID="514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"
Jul 22 01:30:30.817112 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.817099    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4\": container with ID starting with 514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4 not found: ID does not exist" containerID="514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"
Jul 22 01:30:30.817147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817115    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"} err="failed to get container status \"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4\": rpc error: code = NotFound desc = could not find container \"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4\": container with ID starting with 514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4 not found: ID does not exist"
Jul 22 01:30:30.817147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817126    2560 scope.go:117] "RemoveContainer" containerID="562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"
Jul 22 01:30:30.817331 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.817312    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8\": container with ID starting with 562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8 not found: ID does not exist" containerID="562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"
Jul 22 01:30:30.817373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817337    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"} err="failed to get container status \"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8\": rpc error: code = NotFound desc = could not find container \"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8\": container with ID starting with 562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8 not found: ID does not exist"
Jul 22 01:30:30.817373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817352    2560 scope.go:117] "RemoveContainer" containerID="f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"
Jul 22 01:30:30.817599 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.817581    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf\": container with ID starting with f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf not found: ID does not exist" containerID="f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"
Jul 22 01:30:30.817635 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817604    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"} err="failed to get container status \"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf\": rpc error: code = NotFound desc = could not find container \"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf\": container with ID starting with f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf not found: ID does not exist"
Jul 22 01:30:30.817635 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817617    2560 scope.go:117] "RemoveContainer" containerID="3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"
Jul 22 01:30:30.817818 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.817801    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7\": container with ID starting with 3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7 not found: ID does not exist" containerID="3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"
Jul 22 01:30:30.817860 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817822    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"} err="failed to get container status \"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7\": rpc error: code = NotFound desc = could not find container \"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7\": container with ID starting with 3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7 not found: ID does not exist"
Jul 22 01:30:30.817860 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.817835    2560 scope.go:117] "RemoveContainer" containerID="15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"
Jul 22 01:30:30.818010 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:30.817997    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6\": container with ID starting with 15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6 not found: ID does not exist" containerID="15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"
Jul 22 01:30:30.818044 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818015    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"} err="failed to get container status \"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6\": rpc error: code = NotFound desc = could not find container \"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6\": container with ID starting with 15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6 not found: ID does not exist"
Jul 22 01:30:30.818044 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818028    2560 scope.go:117] "RemoveContainer" containerID="7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"
Jul 22 01:30:30.818241 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818224    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac"} err="failed to get container status \"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac\": rpc error: code = NotFound desc = could not find container \"7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac\": container with ID starting with 7b4ce656d9443d6da8995c38ee7047130e897ae3ba939b9873d140c7408a22ac not found: ID does not exist"
Jul 22 01:30:30.818290 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818242    2560 scope.go:117] "RemoveContainer" containerID="9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"
Jul 22 01:30:30.818448 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818429    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6"} err="failed to get container status \"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6\": rpc error: code = NotFound desc = could not find container \"9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6\": container with ID starting with 9defecff7e87b16791b86784ef90884cef6f2227c0a04f30f050917fbcaa6ab6 not found: ID does not exist"
Jul 22 01:30:30.818494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818448    2560 scope.go:117] "RemoveContainer" containerID="514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"
Jul 22 01:30:30.818639 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818621    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4"} err="failed to get container status \"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4\": rpc error: code = NotFound desc = could not find container \"514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4\": container with ID starting with 514c89c8365b205045b42b9b7b0d09e213356ab9069ae8c093aed59fed857ab4 not found: ID does not exist"
Jul 22 01:30:30.818680 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818639    2560 scope.go:117] "RemoveContainer" containerID="562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"
Jul 22 01:30:30.818833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818815    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8"} err="failed to get container status \"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8\": rpc error: code = NotFound desc = could not find container \"562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8\": container with ID starting with 562b8f48d3f9a56141d02fe9cef23562bfeaf495b0534397376c3e48f96311f8 not found: ID does not exist"
Jul 22 01:30:30.818907 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.818835    2560 scope.go:117] "RemoveContainer" containerID="f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"
Jul 22 01:30:30.819050 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.819035    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf"} err="failed to get container status \"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf\": rpc error: code = NotFound desc = could not find container \"f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf\": container with ID starting with f6bb968669fa21bcb7bd61c16e3ca2f92e28d76559dae21882bab84b83b113bf not found: ID does not exist"
Jul 22 01:30:30.819093 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.819050    2560 scope.go:117] "RemoveContainer" containerID="3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"
Jul 22 01:30:30.819231 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.819215    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7"} err="failed to get container status \"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7\": rpc error: code = NotFound desc = could not find container \"3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7\": container with ID starting with 3821c1c695152823c9173e5f558f0b1d5cde7d1c5bec2bf76143c4960c33bcb7 not found: ID does not exist"
Jul 22 01:30:30.819273 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.819232    2560 scope.go:117] "RemoveContainer" containerID="15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"
Jul 22 01:30:30.819451 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:30.819434    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6"} err="failed to get container status \"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6\": rpc error: code = NotFound desc = could not find container \"15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6\": container with ID starting with 15b30507c10b92bc8799239813b82b3cfd62946425bb0b0d0b71f66205b661e6 not found: ID does not exist"
Jul 22 01:30:31.990271 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:31.990239    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" path="/var/lib/kubelet/pods/47e9b3a2-80d7-4a8d-b807-5aef2a1afea4/volumes"
Jul 22 01:30:33.278433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278385    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:33.278898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278753    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="prometheus" containerID="cri-o://5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f" gracePeriod=600
Jul 22 01:30:33.278898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278782    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="thanos-sidecar" containerID="cri-o://8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3" gracePeriod=600
Jul 22 01:30:33.278898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278792    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy" containerID="cri-o://d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f" gracePeriod=600
Jul 22 01:30:33.279054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278864    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-web" containerID="cri-o://9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a" gracePeriod=600
Jul 22 01:30:33.279054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278944    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217" gracePeriod=600
Jul 22 01:30:33.279143 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.278967    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="config-reloader" containerID="cri-o://6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e" gracePeriod=600
Jul 22 01:30:33.516141 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.516119    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.577538 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577463    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.577538 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577517    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.577738 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577549    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.577738 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577576    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.577738 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577608    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.577738 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577635    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578003 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.577882    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.578054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578012    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578068    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw4ds\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578101    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578164    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578193    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578234    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578262    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578289    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578312    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578339    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578367    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578428    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578461    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2\") pod \"490c5963-0bc2-4de9-a509-cbef7fb328d5\" (UID: \"490c5963-0bc2-4de9-a509-cbef7fb328d5\") "
Jul 22 01:30:33.578773 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.578732    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-metrics-client-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.579168 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.579140    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.579739 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.579709    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.579840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.579718    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.579983 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.579960    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.580200 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.580052    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:30:33.580717 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.580654    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config" (OuterVolumeSpecName: "config") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.580807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.580757    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.580897 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.580842    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.581551 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.581519    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.581664 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.581580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.581750 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.581720    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:30:33.582121 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.582098    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.582323 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.582275    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out" (OuterVolumeSpecName: "config-out") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:30:33.582815 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.582776    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds" (OuterVolumeSpecName: "kube-api-access-dw4ds") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "kube-api-access-dw4ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:30:33.582899 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.582845    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:30:33.583261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.583237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.583656 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.583637    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.584035 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.584020    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.591578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.591555    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config" (OuterVolumeSpecName: "web-config") pod "490c5963-0bc2-4de9-a509-cbef7fb328d5" (UID: "490c5963-0bc2-4de9-a509-cbef7fb328d5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:30:33.679279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679230    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-metrics-client-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679270    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679280    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-tls-assets\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679288    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679299    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679307    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679315    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679323    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679331    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679341    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-kube-rbac-proxy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679348    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-grpc-tls\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679356    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679364    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-config\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679372    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-config-out\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679380    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw4ds\" (UniqueName: \"kubernetes.io/projected/490c5963-0bc2-4de9-a509-cbef7fb328d5-kube-api-access-dw4ds\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679389    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/490c5963-0bc2-4de9-a509-cbef7fb328d5-prometheus-k8s-db\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679418    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679426    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-web-config\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.679548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.679434    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/490c5963-0bc2-4de9-a509-cbef7fb328d5-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:30:33.780731 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780696    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217" exitCode=0
Jul 22 01:30:33.780731 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780725    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f" exitCode=0
Jul 22 01:30:33.780731 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780731    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a" exitCode=0
Jul 22 01:30:33.780731 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780736    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3" exitCode=0
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780741    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e" exitCode=0
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780746    2560 generic.go:356] "Generic (PLEG): container finished" podID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f" exitCode=0
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780766    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780776    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"}
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780844    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"}
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"}
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780895    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"}
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780917    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"}
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780922    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.780948 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"}
Jul 22 01:30:33.781360 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.780967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"490c5963-0bc2-4de9-a509-cbef7fb328d5","Type":"ContainerDied","Data":"ea285b72c8cbf36ccf929ed7feb8f0837915c980d6d33a830f66fdc6e7d80152"}
Jul 22 01:30:33.793261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.793240    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.799754 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.799737    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.805523 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.805507    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.810035 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.810017    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:33.812149 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.812127    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.812913 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.812897    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:33.818181 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.818166    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.824537 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.824523    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.830339 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.830323    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.830653 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.830634    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.830718 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.830659    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.830718 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.830677    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.830924 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.830910    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.830958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.830928    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.830958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.830941    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.831120 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.831106    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.831155 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831123    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.831155 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831135    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.831311 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.831295    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.831350 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831314    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.831350 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831325    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.831527 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.831509    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.831590 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831535    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.831590 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831557    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.831748 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.831733    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.831788 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831752    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.831788 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831763    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.831925 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:30:33.831912    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.831963 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831927    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.831963 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.831937    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.832089 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832075    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.832124 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832089    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.832264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832251    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.832264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832263    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.832469 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832446    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.832507 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832472    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.832677 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832661    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.832716 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832678    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.832840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832824    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.832840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832839    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.832988 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832973    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.833033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.832989    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.833133 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833119    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.833174 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833133    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.833331 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833316    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.833370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833331    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.833541 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833524    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.833585 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833541    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.833712 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833692    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.833755 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833714    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.833915 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833897    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.833953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.833917    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.834132 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834117    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.834132 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834132    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.834325 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834305    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.834367 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834328    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.834588 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834569    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.834644 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834589    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.834816 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834792    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.834816 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.834815    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.835070 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835050    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.835138 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835072    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.835305 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835284    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.835349 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835305    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.835553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835536    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.835596 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835553    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.835754 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835738    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.835790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835756    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.835957 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835938    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.835996 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.835958    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.836161 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836144    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.836224 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836162    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.836349 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836332    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.836388 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836350    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.836568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836549    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.836605 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836569    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.836774 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836759    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.836815 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836774    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.836961 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836945    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.837001 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.836961    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.837144 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837129    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.837144 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837143    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.837349 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837331    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.837472 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837350    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.837588 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837575    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.837625 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837588    2560 scope.go:117] "RemoveContainer" containerID="ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"
Jul 22 01:30:33.837790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837774    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217"} err="failed to get container status \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": rpc error: code = NotFound desc = could not find container \"ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217\": container with ID starting with ffca101921f75407c19d7c0902b422bcbab458ea702b5e303e782edec9b43217 not found: ID does not exist"
Jul 22 01:30:33.837841 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837790    2560 scope.go:117] "RemoveContainer" containerID="d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"
Jul 22 01:30:33.837964 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837950    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f"} err="failed to get container status \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": rpc error: code = NotFound desc = could not find container \"d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f\": container with ID starting with d56c0709594e5711bfa4099dd1a67036bc23d2401887764590c7c4bba813005f not found: ID does not exist"
Jul 22 01:30:33.838006 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.837965    2560 scope.go:117] "RemoveContainer" containerID="9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"
Jul 22 01:30:33.838167 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838151    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a"} err="failed to get container status \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": rpc error: code = NotFound desc = could not find container \"9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a\": container with ID starting with 9ef13a75b9fae296e4fbf6eae69c210b0baae499070cbb45b8edb6a6dd49647a not found: ID does not exist"
Jul 22 01:30:33.838207 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838168    2560 scope.go:117] "RemoveContainer" containerID="8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"
Jul 22 01:30:33.838347 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838333    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3"} err="failed to get container status \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": rpc error: code = NotFound desc = could not find container \"8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3\": container with ID starting with 8185b0d65b93e7b7dc07a8f31f14cf7e107bdf99fcf2520f57a2530b3e14fea3 not found: ID does not exist"
Jul 22 01:30:33.838382 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838348    2560 scope.go:117] "RemoveContainer" containerID="6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"
Jul 22 01:30:33.838571 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838555    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e"} err="failed to get container status \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": rpc error: code = NotFound desc = could not find container \"6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e\": container with ID starting with 6891a23b47213e9ebee6258d1720987a7ec5340ce6396b0a83e514c90200a13e not found: ID does not exist"
Jul 22 01:30:33.838618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838571    2560 scope.go:117] "RemoveContainer" containerID="5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"
Jul 22 01:30:33.838780 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838763    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f"} err="failed to get container status \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": rpc error: code = NotFound desc = could not find container \"5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f\": container with ID starting with 5337b451467ee5510185b5e01fc1ceafd5ca882fba27ccc28709d1b213d90e0f not found: ID does not exist"
Jul 22 01:30:33.838826 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838781    2560 scope.go:117] "RemoveContainer" containerID="e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"
Jul 22 01:30:33.839004 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.838987    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122"} err="failed to get container status \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": rpc error: code = NotFound desc = could not find container \"e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122\": container with ID starting with e7fa79cd7b214911cbc0a7b55264899ad1358df78e0311f1e73acc9c24d3e122 not found: ID does not exist"
Jul 22 01:30:33.845103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845084    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:33.845333 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845321    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="init-config-reloader"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845335    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="init-config-reloader"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845345    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="alertmanager"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845350    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="alertmanager"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845356    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="config-reloader"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845362    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="config-reloader"
Jul 22 01:30:33.845373 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845372    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="prom-label-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845377    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="prom-label-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845383    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845388    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845410    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845416    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845422    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845426    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845432    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-metric"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845436    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-metric"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845441    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="prometheus"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845446    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="prometheus"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845451    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845455    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845460    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845465    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845471    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="thanos-sidecar"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845476    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="thanos-sidecar"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845481    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="init-config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845487    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="init-config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845493    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-thanos"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845498    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-thanos"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845534    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845542    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845547    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="kube-rbac-proxy-metric"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845552    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="prom-label-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845557    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="thanos-sidecar"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845562    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="alertmanager"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845566    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="47e9b3a2-80d7-4a8d-b807-5aef2a1afea4" containerName="config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845571    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="config-reloader"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845575    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="prometheus"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845580    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-web"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845585    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy"
Jul 22 01:30:33.845576 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.845590    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" containerName="kube-rbac-proxy-thanos"
Jul 22 01:30:33.850463 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.850439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.853682 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.853665    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 01:30:33.854387 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854374    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 01:30:33.854496 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854477    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 01:30:33.854692 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854677    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 01:30:33.854930 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854906    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 01:30:33.854930 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854912    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 01:30:33.855047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854930    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 01:30:33.855047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.854913    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 01:30:33.855047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.855012    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 01:30:33.856558 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.856376    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 01:30:33.857358 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.857322    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 01:30:33.857463 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.857371    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 01:30:33.857734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.857716    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5ap6ao3rpn71n\""
Jul 22 01:30:33.857809 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.857718    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cclvl\""
Jul 22 01:30:33.860626 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.860610    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 01:30:33.862611 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.862596    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 01:30:33.873215 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.873197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:33.982034 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.981987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982034 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982036    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982062    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982114    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrfqq\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-kube-api-access-xrfqq\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982129    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982143    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-config-out\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982235    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982259    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982292    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982357    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982468    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-web-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.982540 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.982485    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:33.989457 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:33.989430    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="490c5963-0bc2-4de9-a509-cbef7fb328d5" path="/var/lib/kubelet/pods/490c5963-0bc2-4de9-a509-cbef7fb328d5/volumes"
Jul 22 01:30:34.083495 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083384    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083495 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083477    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083665 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083507    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-config-out\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083665 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083624    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083748 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083748 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083698    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083748 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083739    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083801    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083816    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083846    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.083885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083870    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-web-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083896    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083926    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.083950    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084072    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrfqq\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-kube-api-access-xrfqq\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084097    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084170 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084570 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084329    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084620 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084598    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.084861 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.084835    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086218 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.085459    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086453    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086653    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086684    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5516f4a2-0f21-4643-b390-357b8a8f3241-config-out\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086796    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.086850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.086832    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.087842 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.087160    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.087842 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.087560    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.088545 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.088521    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-web-config\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.088961 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.088935    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.089131 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.089112    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.089184 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.089115    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5516f4a2-0f21-4643-b390-357b8a8f3241-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.089580 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.089564    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5516f4a2-0f21-4643-b390-357b8a8f3241-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.095053 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.095035    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrfqq\" (UniqueName: \"kubernetes.io/projected/5516f4a2-0f21-4643-b390-357b8a8f3241-kube-api-access-xrfqq\") pod \"prometheus-k8s-0\" (UID: \"5516f4a2-0f21-4643-b390-357b8a8f3241\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.159773 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.159739    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:30:34.295512 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.295481    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 01:30:34.299042 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:30:34.299008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5516f4a2_0f21_4643_b390_357b8a8f3241.slice/crio-6f3dc5f3781ccf71c48f503750cb16bca315a2f4d9d4f7c97b8420c8b80c908a WatchSource:0}: Error finding container 6f3dc5f3781ccf71c48f503750cb16bca315a2f4d9d4f7c97b8420c8b80c908a: Status 404 returned error can't find the container with id 6f3dc5f3781ccf71c48f503750cb16bca315a2f4d9d4f7c97b8420c8b80c908a
Jul 22 01:30:34.786076 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.786045    2560 generic.go:356] "Generic (PLEG): container finished" podID="5516f4a2-0f21-4643-b390-357b8a8f3241" containerID="2569ca6fe56f200790a3e9a59b86c1b07f91039f73f2737834e64979354959da" exitCode=0
Jul 22 01:30:34.786235 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.786135    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerDied","Data":"2569ca6fe56f200790a3e9a59b86c1b07f91039f73f2737834e64979354959da"}
Jul 22 01:30:34.786235 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:34.786209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"6f3dc5f3781ccf71c48f503750cb16bca315a2f4d9d4f7c97b8420c8b80c908a"}
Jul 22 01:30:35.793552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"08e93690cc1b139110c1b4b4dfa0423f2dd238550d6d3834ceaa6d0ac1aa0ad7"}
Jul 22 01:30:35.793970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"a6d9016798157d65362a39b34b8b86b717f1818af64570669f2801bcb498589d"}
Jul 22 01:30:35.793970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"888b2766bb0897cfedf8d063f81d3cc19a63385767aa1ec23a16cc67771f53c3"}
Jul 22 01:30:35.793970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793645    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"bd35c5be3c443e67b3e32d90d898218f7c866a374c5c1aaaf94b40ab59aac448"}
Jul 22 01:30:35.793970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793676    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"d8e20a03c1b794fe5d8093ed158ca6f068dd697ffddb512ff9cbbd6e6327ac86"}
Jul 22 01:30:35.793970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.793699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5516f4a2-0f21-4643-b390-357b8a8f3241","Type":"ContainerStarted","Data":"25e7f3b0d415891d12f006d8dd6d564e9844017556524f4cd17f062b1a2561b4"}
Jul 22 01:30:35.823979 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:35.823920    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.805894496 podStartE2EDuration="2.823899875s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.005379ms" isStatefulPod=false podCreationTimestamp="2026-07-22 01:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:30:35.822212885 +0000 UTC m=+154.370631045" watchObservedRunningTime="2026-07-22 01:30:35.823899875 +0000 UTC m=+154.372317999"
Jul 22 01:30:39.160525 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:30:39.160479    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:31:34.160106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:31:34.160075    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:31:34.174980 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:31:34.174955    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:31:35.007411 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:31:35.007378    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 01:33:01.824084 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:01.824056    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 01:33:37.367491 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.367458    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"]
Jul 22 01:33:37.370701 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.370684    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.373265 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.373242    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 01:33:37.374150 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.374130    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 01:33:37.374302 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.374275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jfscj\""
Jul 22 01:33:37.383359 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.383336    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"]
Jul 22 01:33:37.393332 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.393308    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/697861bd-2c07-4fd2-ba03-8355f1f43a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.393332 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.393336    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49xcp\" (UniqueName: \"kubernetes.io/projected/697861bd-2c07-4fd2-ba03-8355f1f43a73-kube-api-access-49xcp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.494052 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.494019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/697861bd-2c07-4fd2-ba03-8355f1f43a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.494052 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.494050    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49xcp\" (UniqueName: \"kubernetes.io/projected/697861bd-2c07-4fd2-ba03-8355f1f43a73-kube-api-access-49xcp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.494380 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.494361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/697861bd-2c07-4fd2-ba03-8355f1f43a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.503184 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.503158    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49xcp\" (UniqueName: \"kubernetes.io/projected/697861bd-2c07-4fd2-ba03-8355f1f43a73-kube-api-access-49xcp\") pod \"openshift-lws-operator-fd8ccff4c-wpcfc\" (UID: \"697861bd-2c07-4fd2-ba03-8355f1f43a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.689091 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.689009    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"
Jul 22 01:33:37.823649 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.823620    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc"]
Jul 22 01:33:37.827014 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:33:37.826989    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod697861bd_2c07_4fd2_ba03_8355f1f43a73.slice/crio-fa05976e371f08a5410b397de39a09b5cb277816f651741068a7939b23b93302 WatchSource:0}: Error finding container fa05976e371f08a5410b397de39a09b5cb277816f651741068a7939b23b93302: Status 404 returned error can't find the container with id fa05976e371f08a5410b397de39a09b5cb277816f651741068a7939b23b93302
Jul 22 01:33:37.828494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:37.828476    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 01:33:38.408081 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:38.408039    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc" event={"ID":"697861bd-2c07-4fd2-ba03-8355f1f43a73","Type":"ContainerStarted","Data":"fa05976e371f08a5410b397de39a09b5cb277816f651741068a7939b23b93302"}
Jul 22 01:33:41.426591 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:41.426549    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc" event={"ID":"697861bd-2c07-4fd2-ba03-8355f1f43a73","Type":"ContainerStarted","Data":"c1530d1c45f1bccaed16cf359e01c753bcff4e27f7d3767d7a9ecd80ac3f4e07"}
Jul 22 01:33:41.446523 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:41.446475    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wpcfc" podStartSLOduration=1.714030321 podStartE2EDuration="4.446456766s" totalImagesPullingTime="2.732426445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:33:41.445217178 +0000 UTC m=+339.993635300" watchObservedRunningTime="2026-07-22 01:33:41.446456766 +0000 UTC m=+339.994874888"
Jul 22 01:33:58.162933 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.162857    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"]
Jul 22 01:33:58.167302 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.167281    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.170348 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.170325    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x7rff\""
Jul 22 01:33:58.170484 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.170376    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 01:33:58.170484 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.170467    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 01:33:58.170567 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.170525    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 01:33:58.171193 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.171176    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 01:33:58.183188 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.183157    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"]
Jul 22 01:33:58.262412 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.262364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-tmp\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.262575 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.262439    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.262575 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.262468    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-webhook-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.262575 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.262507    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b27gz\" (UniqueName: \"kubernetes.io/projected/b36749e4-1826-418f-bf79-ebba86f6418e-kube-api-access-b27gz\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.262690 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.262570    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-manifests\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364038 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.363996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364038 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.364036    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-webhook-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364245 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.364165    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b27gz\" (UniqueName: \"kubernetes.io/projected/b36749e4-1826-418f-bf79-ebba86f6418e-kube-api-access-b27gz\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364245 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.364219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-manifests\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364312 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.364264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-tmp\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.364671 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.364638    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-manifests\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.366486 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.366465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36749e4-1826-418f-bf79-ebba86f6418e-tmp\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.366599 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.366584    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.366692 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.366673    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b36749e4-1826-418f-bf79-ebba86f6418e-webhook-cert\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.372730 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.372707    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b27gz\" (UniqueName: \"kubernetes.io/projected/b36749e4-1826-418f-bf79-ebba86f6418e-kube-api-access-b27gz\") pod \"opendatahub-operator-controller-manager-9869b8c7c-nppzp\" (UID: \"b36749e4-1826-418f-bf79-ebba86f6418e\") " pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.477488 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.477378    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:33:58.628687 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:58.628653    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"]
Jul 22 01:33:58.631694 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:33:58.631668    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36749e4_1826_418f_bf79_ebba86f6418e.slice/crio-26761383308c993855692a99373d47605928e1c35fa93261215142dce17c9d57 WatchSource:0}: Error finding container 26761383308c993855692a99373d47605928e1c35fa93261215142dce17c9d57: Status 404 returned error can't find the container with id 26761383308c993855692a99373d47605928e1c35fa93261215142dce17c9d57
Jul 22 01:33:59.490261 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:33:59.490158    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp" event={"ID":"b36749e4-1826-418f-bf79-ebba86f6418e","Type":"ContainerStarted","Data":"26761383308c993855692a99373d47605928e1c35fa93261215142dce17c9d57"}
Jul 22 01:34:02.503743 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:02.503710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp" event={"ID":"b36749e4-1826-418f-bf79-ebba86f6418e","Type":"ContainerStarted","Data":"7da1ff37688066cd098d31d562d92b6e705f866286ae18947c389fa66c2cbed4"}
Jul 22 01:34:03.509001 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:03.508966    2560 generic.go:356] "Generic (PLEG): container finished" podID="b36749e4-1826-418f-bf79-ebba86f6418e" containerID="7da1ff37688066cd098d31d562d92b6e705f866286ae18947c389fa66c2cbed4" exitCode=0
Jul 22 01:34:03.509485 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:03.509038    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp" event={"ID":"b36749e4-1826-418f-bf79-ebba86f6418e","Type":"ContainerDied","Data":"7da1ff37688066cd098d31d562d92b6e705f866286ae18947c389fa66c2cbed4"}
Jul 22 01:34:04.514687 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:04.514649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp" event={"ID":"b36749e4-1826-418f-bf79-ebba86f6418e","Type":"ContainerStarted","Data":"c7544410dec9e8771f8c5b9413608f51cac08aa691cb34eb8f0fbeb1feb0ae39"}
Jul 22 01:34:04.515083 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:04.514773    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:34:04.549762 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:04.549704    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp" podStartSLOduration=2.165449992 podStartE2EDuration="6.549687426s" totalImagesPullingTime="3.290389983s" totalInitContainerRuntime="1.093847451s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:33:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:34:04.548905695 +0000 UTC m=+363.097323818" watchObservedRunningTime="2026-07-22 01:34:04.549687426 +0000 UTC m=+363.098105548"
Jul 22 01:34:15.523178 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:15.523138    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9869b8c7c-nppzp"
Jul 22 01:34:18.306967 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.306931    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hg6hf"]
Jul 22 01:34:18.314556 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.314537    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.318483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.318463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xfcnq\""
Jul 22 01:34:18.318594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.318463    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 01:34:18.330549 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.330524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hg6hf"]
Jul 22 01:34:18.431263 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.431226    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3721b4d-0d8f-42da-b84c-751b3d094475-cert\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.431263 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.431266    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvtdv\" (UniqueName: \"kubernetes.io/projected/b3721b4d-0d8f-42da-b84c-751b3d094475-kube-api-access-jvtdv\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.532640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.532597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3721b4d-0d8f-42da-b84c-751b3d094475-cert\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.532640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.532642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvtdv\" (UniqueName: \"kubernetes.io/projected/b3721b4d-0d8f-42da-b84c-751b3d094475-kube-api-access-jvtdv\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.535008 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.534985    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3721b4d-0d8f-42da-b84c-751b3d094475-cert\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.541450 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.541376    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvtdv\" (UniqueName: \"kubernetes.io/projected/b3721b4d-0d8f-42da-b84c-751b3d094475-kube-api-access-jvtdv\") pod \"odh-model-controller-7998c67dc4-hg6hf\" (UID: \"b3721b4d-0d8f-42da-b84c-751b3d094475\") " pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.623990 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.623961    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:18.750362 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:18.750334    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hg6hf"]
Jul 22 01:34:18.753175 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:34:18.753147    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3721b4d_0d8f_42da_b84c_751b3d094475.slice/crio-6918ebc4a1f4ce6bd05c9f8482780ed7541ee17e6427284a2ca58d0069f17d7e WatchSource:0}: Error finding container 6918ebc4a1f4ce6bd05c9f8482780ed7541ee17e6427284a2ca58d0069f17d7e: Status 404 returned error can't find the container with id 6918ebc4a1f4ce6bd05c9f8482780ed7541ee17e6427284a2ca58d0069f17d7e
Jul 22 01:34:19.575737 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:19.575696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf" event={"ID":"b3721b4d-0d8f-42da-b84c-751b3d094475","Type":"ContainerStarted","Data":"6918ebc4a1f4ce6bd05c9f8482780ed7541ee17e6427284a2ca58d0069f17d7e"}
Jul 22 01:34:21.585486 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:21.585454    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf" event={"ID":"b3721b4d-0d8f-42da-b84c-751b3d094475","Type":"ContainerStarted","Data":"6573094a534eeb426d29c5b565ec9f855b6f19614c810c10746324aa4f4b2e8a"}
Jul 22 01:34:21.585829 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:21.585588    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:21.606812 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:21.606764    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf" podStartSLOduration=0.841666451 podStartE2EDuration="3.606749937s" totalImagesPullingTime="2.765083486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:34:21.606460481 +0000 UTC m=+380.154878601" watchObservedRunningTime="2026-07-22 01:34:21.606749937 +0000 UTC m=+380.155168059"
Jul 22 01:34:25.027772 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.027737    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"]
Jul 22 01:34:25.030748 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.030734    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.033472 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.033449    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 01:34:25.033595 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.033451    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 01:34:25.034166 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.034148    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6mlcj\""
Jul 22 01:34:25.034277 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.034203    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 01:34:25.041440 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.041418    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"]
Jul 22 01:34:25.188362 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.188325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0ccc9b1-4473-4013-93bf-05b13150ac86-manager-config\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.188568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.188384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ssgl\" (UniqueName: \"kubernetes.io/projected/a0ccc9b1-4473-4013-93bf-05b13150ac86-kube-api-access-9ssgl\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.188568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.188460    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.188568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.188490    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-metrics-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.289094 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.289011    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-metrics-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.289094 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.289065    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0ccc9b1-4473-4013-93bf-05b13150ac86-manager-config\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.289274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.289117    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ssgl\" (UniqueName: \"kubernetes.io/projected/a0ccc9b1-4473-4013-93bf-05b13150ac86-kube-api-access-9ssgl\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.289274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.289156    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.289798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.289780    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0ccc9b1-4473-4013-93bf-05b13150ac86-manager-config\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.291516 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.291494    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-metrics-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.291653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.291634    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0ccc9b1-4473-4013-93bf-05b13150ac86-cert\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.298975 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.298952    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ssgl\" (UniqueName: \"kubernetes.io/projected/a0ccc9b1-4473-4013-93bf-05b13150ac86-kube-api-access-9ssgl\") pod \"lws-controller-manager-7898dc9f59-qbjf6\" (UID: \"a0ccc9b1-4473-4013-93bf-05b13150ac86\") " pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.339943 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.339916    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:25.469798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.469757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"]
Jul 22 01:34:25.472673 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:34:25.472649    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ccc9b1_4473_4013_93bf_05b13150ac86.slice/crio-6e8519cc7b68ae4caeb508ed56370b476839480ac7fce2ef9bf2546505df4bd3 WatchSource:0}: Error finding container 6e8519cc7b68ae4caeb508ed56370b476839480ac7fce2ef9bf2546505df4bd3: Status 404 returned error can't find the container with id 6e8519cc7b68ae4caeb508ed56370b476839480ac7fce2ef9bf2546505df4bd3
Jul 22 01:34:25.607593 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:25.607563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6" event={"ID":"a0ccc9b1-4473-4013-93bf-05b13150ac86","Type":"ContainerStarted","Data":"6e8519cc7b68ae4caeb508ed56370b476839480ac7fce2ef9bf2546505df4bd3"}
Jul 22 01:34:27.617290 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:27.617249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6" event={"ID":"a0ccc9b1-4473-4013-93bf-05b13150ac86","Type":"ContainerStarted","Data":"9bf8b7321d5c8b1a4d513903265c0ad65399989f13d6fef7d453fbfb436d2eed"}
Jul 22 01:34:27.617681 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:27.617376    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:34:28.632031 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.631981    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6" podStartSLOduration=1.815986321 podStartE2EDuration="3.631964467s" totalImagesPullingTime="1.815978146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:34:27.643153476 +0000 UTC m=+386.191571600" watchObservedRunningTime="2026-07-22 01:34:28.631964467 +0000 UTC m=+387.180382589"
Jul 22 01:34:28.632548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.632533    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"]
Jul 22 01:34:28.635653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.635632    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.644335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.644315    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 01:34:28.644800 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.644785    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 01:34:28.644849 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.644826    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 01:34:28.645105 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.645089    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 01:34:28.646233 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.646218    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zfl9j\""
Jul 22 01:34:28.707833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.707801    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"]
Jul 22 01:34:28.819341 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.819287    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx8q7\" (UniqueName: \"kubernetes.io/projected/aedf98a0-caa4-4d30-acbc-466eed9bdc84-kube-api-access-rx8q7\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.819543 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.819370    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tls-certs\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.819543 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.819431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tmp\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.919968 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.919886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tls-certs\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.920323 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.920283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tmp\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.920862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.920434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx8q7\" (UniqueName: \"kubernetes.io/projected/aedf98a0-caa4-4d30-acbc-466eed9bdc84-kube-api-access-rx8q7\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.928813 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.928545    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tls-certs\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.929279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.929254    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aedf98a0-caa4-4d30-acbc-466eed9bdc84-tmp\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.931311 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.931286    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx8q7\" (UniqueName: \"kubernetes.io/projected/aedf98a0-caa4-4d30-acbc-466eed9bdc84-kube-api-access-rx8q7\") pod \"kube-auth-proxy-5d7dbc5fd-98jlr\" (UID: \"aedf98a0-caa4-4d30-acbc-466eed9bdc84\") " pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:28.944734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:28.944709    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"
Jul 22 01:34:29.078001 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:29.077971    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr"]
Jul 22 01:34:29.080208 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:34:29.080179    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaedf98a0_caa4_4d30_acbc_466eed9bdc84.slice/crio-a54b98d42866bb2f4fdf129687fd2e3e571eddab99d698522e4a1f0f19292156 WatchSource:0}: Error finding container a54b98d42866bb2f4fdf129687fd2e3e571eddab99d698522e4a1f0f19292156: Status 404 returned error can't find the container with id a54b98d42866bb2f4fdf129687fd2e3e571eddab99d698522e4a1f0f19292156
Jul 22 01:34:29.626345 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:29.626309    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr" event={"ID":"aedf98a0-caa4-4d30-acbc-466eed9bdc84","Type":"ContainerStarted","Data":"a54b98d42866bb2f4fdf129687fd2e3e571eddab99d698522e4a1f0f19292156"}
Jul 22 01:34:32.591994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:32.591909    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hg6hf"
Jul 22 01:34:32.641076 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:32.641045    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr" event={"ID":"aedf98a0-caa4-4d30-acbc-466eed9bdc84","Type":"ContainerStarted","Data":"080f5fb0e6ef3fcad9624d857a20c6a9d52384be1fc618903c35f281f2e89492"}
Jul 22 01:34:32.686034 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:32.685972    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d7dbc5fd-98jlr" podStartSLOduration=1.439289702 podStartE2EDuration="4.685946998s" totalImagesPullingTime="3.246657296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:34:32.684686965 +0000 UTC m=+391.233105088" watchObservedRunningTime="2026-07-22 01:34:32.685946998 +0000 UTC m=+391.234365145"
Jul 22 01:34:38.623688 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:34:38.623656    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7898dc9f59-qbjf6"
Jul 22 01:35:59.321750 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.321716    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"]
Jul 22 01:35:59.324898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.324882    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:35:59.328374 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.328354    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5w9k8\""
Jul 22 01:35:59.328499 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.328385    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 01:35:59.329117 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.329102    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 01:35:59.329215 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.329197    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 01:35:59.339633 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.339612    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"]
Jul 22 01:35:59.461694 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.461656    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktbd5\" (UniqueName: \"kubernetes.io/projected/cb9818f7-8f7a-4963-b148-1fc73d3bbb7f-kube-api-access-ktbd5\") pod \"dns-operator-controller-manager-864dd78c4c-gd7sm\" (UID: \"cb9818f7-8f7a-4963-b148-1fc73d3bbb7f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:35:59.562938 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.562858    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktbd5\" (UniqueName: \"kubernetes.io/projected/cb9818f7-8f7a-4963-b148-1fc73d3bbb7f-kube-api-access-ktbd5\") pod \"dns-operator-controller-manager-864dd78c4c-gd7sm\" (UID: \"cb9818f7-8f7a-4963-b148-1fc73d3bbb7f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:35:59.575721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.575690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktbd5\" (UniqueName: \"kubernetes.io/projected/cb9818f7-8f7a-4963-b148-1fc73d3bbb7f-kube-api-access-ktbd5\") pod \"dns-operator-controller-manager-864dd78c4c-gd7sm\" (UID: \"cb9818f7-8f7a-4963-b148-1fc73d3bbb7f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:35:59.634474 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.634441    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:35:59.773347 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.773316    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"]
Jul 22 01:35:59.775766 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:35:59.775741    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9818f7_8f7a_4963_b148_1fc73d3bbb7f.slice/crio-ed364ee79d6b5b2214522a9531b16205eb7383e72dcc3cb672612184f77aa21e WatchSource:0}: Error finding container ed364ee79d6b5b2214522a9531b16205eb7383e72dcc3cb672612184f77aa21e: Status 404 returned error can't find the container with id ed364ee79d6b5b2214522a9531b16205eb7383e72dcc3cb672612184f77aa21e
Jul 22 01:35:59.992869 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:35:59.992831    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm" event={"ID":"cb9818f7-8f7a-4963-b148-1fc73d3bbb7f","Type":"ContainerStarted","Data":"ed364ee79d6b5b2214522a9531b16205eb7383e72dcc3cb672612184f77aa21e"}
Jul 22 01:36:02.130097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:02.130072    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 01:36:03.007581 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:03.007540    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm" event={"ID":"cb9818f7-8f7a-4963-b148-1fc73d3bbb7f","Type":"ContainerStarted","Data":"b25537821eb2b725e68279ff606483b6764dc9844a0285bdf8d1cbd38c02024b"}
Jul 22 01:36:03.007766 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:03.007629    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:36:03.029607 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:03.029556    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm" podStartSLOduration=1.680912351 podStartE2EDuration="4.029540144s" totalImagesPullingTime="2.348627793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:36:03.027556502 +0000 UTC m=+481.575974785" watchObservedRunningTime="2026-07-22 01:36:03.029540144 +0000 UTC m=+481.577958267"
Jul 22 01:36:05.769225 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.769191    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"]
Jul 22 01:36:05.771702 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.771686    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:05.775420 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.775384    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 01:36:05.775763 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.775744    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 01:36:05.776290 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.776275    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-b826r\""
Jul 22 01:36:05.783868 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.783847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"]
Jul 22 01:36:05.918253 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.918216    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e023244e-57b5-4967-83d0-2f3728285350-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:05.918253 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.918257    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e023244e-57b5-4967-83d0-2f3728285350-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:05.918494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:05.918316    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdkkh\" (UniqueName: \"kubernetes.io/projected/e023244e-57b5-4967-83d0-2f3728285350-kube-api-access-tdkkh\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.019029 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.018995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e023244e-57b5-4967-83d0-2f3728285350-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.019029 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.019033    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e023244e-57b5-4967-83d0-2f3728285350-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.019312 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.019161    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdkkh\" (UniqueName: \"kubernetes.io/projected/e023244e-57b5-4967-83d0-2f3728285350-kube-api-access-tdkkh\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.019625 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.019609    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e023244e-57b5-4967-83d0-2f3728285350-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.021410 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.021373    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e023244e-57b5-4967-83d0-2f3728285350-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.027947 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.027918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdkkh\" (UniqueName: \"kubernetes.io/projected/e023244e-57b5-4967-83d0-2f3728285350-kube-api-access-tdkkh\") pod \"kuadrant-console-plugin-66b5774985-9vb8n\" (UID: \"e023244e-57b5-4967-83d0-2f3728285350\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.079922 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.079884    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"
Jul 22 01:36:06.210230 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:06.210197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n"]
Jul 22 01:36:06.214112 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:36:06.214089    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode023244e_57b5_4967_83d0_2f3728285350.slice/crio-c2a2f4d647bd9a93605fa2bf51b3de67ab7421fce50620907ce26311531f67cd WatchSource:0}: Error finding container c2a2f4d647bd9a93605fa2bf51b3de67ab7421fce50620907ce26311531f67cd: Status 404 returned error can't find the container with id c2a2f4d647bd9a93605fa2bf51b3de67ab7421fce50620907ce26311531f67cd
Jul 22 01:36:07.024480 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:07.024447    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n" event={"ID":"e023244e-57b5-4967-83d0-2f3728285350","Type":"ContainerStarted","Data":"c2a2f4d647bd9a93605fa2bf51b3de67ab7421fce50620907ce26311531f67cd"}
Jul 22 01:36:14.014666 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:14.014624    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gd7sm"
Jul 22 01:36:29.132837 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:29.132801    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n" event={"ID":"e023244e-57b5-4967-83d0-2f3728285350","Type":"ContainerStarted","Data":"1c4bd85df6c29b2377d7a80cfe39b789692dcc6133b63ed1613ac8fa230ff696"}
Jul 22 01:36:29.163550 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:29.163503    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9vb8n" podStartSLOduration=2.0073121 podStartE2EDuration="24.163486722s" totalImagesPullingTime="22.156174622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:36:29.161825626 +0000 UTC m=+507.710243746" watchObservedRunningTime="2026-07-22 01:36:29.163486722 +0000 UTC m=+507.711904883"
Jul 22 01:36:47.792642 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.792609    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:36:47.794863 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.794846    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:47.797121 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.797096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 01:36:47.809153 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.809130    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:36:47.891755 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.891723    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:47.892072 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.892042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj72j\" (UniqueName: \"kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:47.893152 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.893131    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:36:47.992734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.992697    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj72j\" (UniqueName: \"kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:47.992909 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.992755    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:47.993359 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:47.993342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:48.001188 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.001163    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj72j\" (UniqueName: \"kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j\") pod \"limitador-limitador-64c94fd64f-phjmq\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:48.103960 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.103867    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:48.236657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.236621    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:36:48.240119 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:36:48.240095    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8aeb6f_71ec_4168_821b_69d4a3aea371.slice/crio-74982bc928fa8ae2deaebf0d26facdb12d283b4d0190caccdc91a7de617a8255 WatchSource:0}: Error finding container 74982bc928fa8ae2deaebf0d26facdb12d283b4d0190caccdc91a7de617a8255: Status 404 returned error can't find the container with id 74982bc928fa8ae2deaebf0d26facdb12d283b4d0190caccdc91a7de617a8255
Jul 22 01:36:48.609212 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.609178    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:48.612074 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.612058    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:48.615815 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.615797    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cwvnk\""
Jul 22 01:36:48.626599 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.626577    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:48.699976 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.699940    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f5cx\" (UniqueName: \"kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx\") pod \"authorino-58cd955675-6pzzw\" (UID: \"8cba1b66-3aef-4273-ad65-99ab3b551abc\") " pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:48.800658 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.800620    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f5cx\" (UniqueName: \"kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx\") pod \"authorino-58cd955675-6pzzw\" (UID: \"8cba1b66-3aef-4273-ad65-99ab3b551abc\") " pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:48.811888 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.811853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f5cx\" (UniqueName: \"kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx\") pod \"authorino-58cd955675-6pzzw\" (UID: \"8cba1b66-3aef-4273-ad65-99ab3b551abc\") " pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:48.920765 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:48.920683    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:49.047740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:49.047710    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:49.050931 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:36:49.050898    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cba1b66_3aef_4273_ad65_99ab3b551abc.slice/crio-a2d53cce65facf8d6714a56c7cea27d652fcbabc256fa6a3329b5a4366dcc303 WatchSource:0}: Error finding container a2d53cce65facf8d6714a56c7cea27d652fcbabc256fa6a3329b5a4366dcc303: Status 404 returned error can't find the container with id a2d53cce65facf8d6714a56c7cea27d652fcbabc256fa6a3329b5a4366dcc303
Jul 22 01:36:49.216179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:49.216098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" event={"ID":"7f8aeb6f-71ec-4168-821b-69d4a3aea371","Type":"ContainerStarted","Data":"74982bc928fa8ae2deaebf0d26facdb12d283b4d0190caccdc91a7de617a8255"}
Jul 22 01:36:49.217201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:49.217177    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6pzzw" event={"ID":"8cba1b66-3aef-4273-ad65-99ab3b551abc","Type":"ContainerStarted","Data":"a2d53cce65facf8d6714a56c7cea27d652fcbabc256fa6a3329b5a4366dcc303"}
Jul 22 01:36:53.815355 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:53.815253    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:54.243240 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:54.243207    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" event={"ID":"7f8aeb6f-71ec-4168-821b-69d4a3aea371","Type":"ContainerStarted","Data":"64023e83e8287a220f8b2c50c6057867563df6150762d49f504f36616db6ce9e"}
Jul 22 01:36:54.243474 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:54.243357    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:36:54.244684 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:54.244662    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6pzzw" event={"ID":"8cba1b66-3aef-4273-ad65-99ab3b551abc","Type":"ContainerStarted","Data":"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"}
Jul 22 01:36:54.264095 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:54.264048    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" podStartSLOduration=2.116030918 podStartE2EDuration="7.264032048s" totalImagesPullingTime="5.14800113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:36:54.263646838 +0000 UTC m=+532.812064960" watchObservedRunningTime="2026-07-22 01:36:54.264032048 +0000 UTC m=+532.812450170"
Jul 22 01:36:54.279881 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:54.279837    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6pzzw" podStartSLOduration=1.94251062 podStartE2EDuration="6.279818532s" totalImagesPullingTime="4.337307912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:36:54.279255681 +0000 UTC m=+532.827673803" watchObservedRunningTime="2026-07-22 01:36:54.279818532 +0000 UTC m=+532.828236705"
Jul 22 01:36:55.248621 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:55.248582    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6pzzw" podUID="8cba1b66-3aef-4273-ad65-99ab3b551abc" containerName="authorino" containerID="cri-o://66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691" gracePeriod=30
Jul 22 01:36:55.495201 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:55.495176    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:55.663559 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:55.663529    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f5cx\" (UniqueName: \"kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx\") pod \"8cba1b66-3aef-4273-ad65-99ab3b551abc\" (UID: \"8cba1b66-3aef-4273-ad65-99ab3b551abc\") "
Jul 22 01:36:55.665630 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:55.665599    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx" (OuterVolumeSpecName: "kube-api-access-2f5cx") pod "8cba1b66-3aef-4273-ad65-99ab3b551abc" (UID: "8cba1b66-3aef-4273-ad65-99ab3b551abc"). InnerVolumeSpecName "kube-api-access-2f5cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:36:55.764474 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:55.764437    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f5cx\" (UniqueName: \"kubernetes.io/projected/8cba1b66-3aef-4273-ad65-99ab3b551abc-kube-api-access-2f5cx\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:36:56.253609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.253525    2560 generic.go:356] "Generic (PLEG): container finished" podID="8cba1b66-3aef-4273-ad65-99ab3b551abc" containerID="66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691" exitCode=0
Jul 22 01:36:56.253609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.253565    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6pzzw"
Jul 22 01:36:56.253609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.253571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6pzzw" event={"ID":"8cba1b66-3aef-4273-ad65-99ab3b551abc","Type":"ContainerDied","Data":"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"}
Jul 22 01:36:56.254078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.253619    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6pzzw" event={"ID":"8cba1b66-3aef-4273-ad65-99ab3b551abc","Type":"ContainerDied","Data":"a2d53cce65facf8d6714a56c7cea27d652fcbabc256fa6a3329b5a4366dcc303"}
Jul 22 01:36:56.254078 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.253642    2560 scope.go:117] "RemoveContainer" containerID="66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"
Jul 22 01:36:56.261594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.261573    2560 scope.go:117] "RemoveContainer" containerID="66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"
Jul 22 01:36:56.261889 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:36:56.261866    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691\": container with ID starting with 66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691 not found: ID does not exist" containerID="66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"
Jul 22 01:36:56.262031 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.261901    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691"} err="failed to get container status \"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691\": rpc error: code = NotFound desc = could not find container \"66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691\": container with ID starting with 66248bc335ac19238e59ba8aae75e02ace5c0a9cc15223248a366aff92fb5691 not found: ID does not exist"
Jul 22 01:36:56.287964 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.287928    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:56.295328 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:56.295297    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6pzzw"]
Jul 22 01:36:57.990262 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:36:57.990230    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cba1b66-3aef-4273-ad65-99ab3b551abc" path="/var/lib/kubelet/pods/8cba1b66-3aef-4273-ad65-99ab3b551abc/volumes"
Jul 22 01:37:04.836365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:04.836330    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:37:04.836842 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:04.836527    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" podUID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" containerName="limitador" containerID="cri-o://64023e83e8287a220f8b2c50c6057867563df6150762d49f504f36616db6ce9e" gracePeriod=30
Jul 22 01:37:04.837222 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:04.837204    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:37:05.293164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.293130    2560 generic.go:356] "Generic (PLEG): container finished" podID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" containerID="64023e83e8287a220f8b2c50c6057867563df6150762d49f504f36616db6ce9e" exitCode=0
Jul 22 01:37:05.293319 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.293212    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" event={"ID":"7f8aeb6f-71ec-4168-821b-69d4a3aea371","Type":"ContainerDied","Data":"64023e83e8287a220f8b2c50c6057867563df6150762d49f504f36616db6ce9e"}
Jul 22 01:37:05.391708 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.391685    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:37:05.451995 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.451906    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file\") pod \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") "
Jul 22 01:37:05.451995 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.451937    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj72j\" (UniqueName: \"kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j\") pod \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\" (UID: \"7f8aeb6f-71ec-4168-821b-69d4a3aea371\") "
Jul 22 01:37:05.452249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.452223    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file" (OuterVolumeSpecName: "config-file") pod "7f8aeb6f-71ec-4168-821b-69d4a3aea371" (UID: "7f8aeb6f-71ec-4168-821b-69d4a3aea371"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:37:05.453915 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.453896    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j" (OuterVolumeSpecName: "kube-api-access-rj72j") pod "7f8aeb6f-71ec-4168-821b-69d4a3aea371" (UID: "7f8aeb6f-71ec-4168-821b-69d4a3aea371"). InnerVolumeSpecName "kube-api-access-rj72j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:37:05.552517 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.552464    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7f8aeb6f-71ec-4168-821b-69d4a3aea371-config-file\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:37:05.552517 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:05.552506    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj72j\" (UniqueName: \"kubernetes.io/projected/7f8aeb6f-71ec-4168-821b-69d4a3aea371-kube-api-access-rj72j\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:37:06.298639 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:06.298552    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq" event={"ID":"7f8aeb6f-71ec-4168-821b-69d4a3aea371","Type":"ContainerDied","Data":"74982bc928fa8ae2deaebf0d26facdb12d283b4d0190caccdc91a7de617a8255"}
Jul 22 01:37:06.298639 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:06.298606    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-phjmq"
Jul 22 01:37:06.298639 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:06.298618    2560 scope.go:117] "RemoveContainer" containerID="64023e83e8287a220f8b2c50c6057867563df6150762d49f504f36616db6ce9e"
Jul 22 01:37:06.321070 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:06.321041    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:37:06.325899 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:06.325876    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-phjmq"]
Jul 22 01:37:07.989841 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:07.989807    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" path="/var/lib/kubelet/pods/7f8aeb6f-71ec-4168-821b-69d4a3aea371/volumes"
Jul 22 01:37:09.671079 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671049    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"]
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671341    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" containerName="limitador"
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671352    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" containerName="limitador"
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671361    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cba1b66-3aef-4273-ad65-99ab3b551abc" containerName="authorino"
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671366    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cba1b66-3aef-4273-ad65-99ab3b551abc" containerName="authorino"
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671433    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8aeb6f-71ec-4168-821b-69d4a3aea371" containerName="limitador"
Jul 22 01:37:09.671509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.671442    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cba1b66-3aef-4273-ad65-99ab3b551abc" containerName="authorino"
Jul 22 01:37:09.675610 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.675595    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.679105 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.679082    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 22 01:37:09.679982 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.679966    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-q6lnq\""
Jul 22 01:37:09.680202 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.680182    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 01:37:09.684707 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.684692    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 01:37:09.691012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.690980    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgzt9\" (UniqueName: \"kubernetes.io/projected/6529147f-5bbf-4344-b6f7-794a610329c6-kube-api-access-cgzt9\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.691102 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.691079    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6529147f-5bbf-4344-b6f7-794a610329c6-data\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.694013 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.693995    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"]
Jul 22 01:37:09.792043 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.792004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgzt9\" (UniqueName: \"kubernetes.io/projected/6529147f-5bbf-4344-b6f7-794a610329c6-kube-api-access-cgzt9\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.792249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.792090    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6529147f-5bbf-4344-b6f7-794a610329c6-data\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.792536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.792515    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6529147f-5bbf-4344-b6f7-794a610329c6-data\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.814725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.814696    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgzt9\" (UniqueName: \"kubernetes.io/projected/6529147f-5bbf-4344-b6f7-794a610329c6-kube-api-access-cgzt9\") pod \"postgres-69fb4594cf-fjdlg\" (UID: \"6529147f-5bbf-4344-b6f7-794a610329c6\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:09.985010 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:09.984918    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:10.112376 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:10.112354    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"]
Jul 22 01:37:10.114849 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:37:10.114818    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6529147f_5bbf_4344_b6f7_794a610329c6.slice/crio-45cfb12e56cbcb1e597191b9ef1be69317c1b37ffbe7ed4e6ff329459e939335 WatchSource:0}: Error finding container 45cfb12e56cbcb1e597191b9ef1be69317c1b37ffbe7ed4e6ff329459e939335: Status 404 returned error can't find the container with id 45cfb12e56cbcb1e597191b9ef1be69317c1b37ffbe7ed4e6ff329459e939335
Jul 22 01:37:10.317355 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:10.317270    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg" event={"ID":"6529147f-5bbf-4344-b6f7-794a610329c6","Type":"ContainerStarted","Data":"45cfb12e56cbcb1e597191b9ef1be69317c1b37ffbe7ed4e6ff329459e939335"}
Jul 22 01:37:16.346157 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:16.346115    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg" event={"ID":"6529147f-5bbf-4344-b6f7-794a610329c6","Type":"ContainerStarted","Data":"9ef82fa960fca949df7663dfde3c53e73a9471a948f4ddb46bdbb188982613fc"}
Jul 22 01:37:16.346629 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:16.346185    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:16.365437 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:16.365362    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg" podStartSLOduration=1.757880189 podStartE2EDuration="7.365347623s" totalImagesPullingTime="5.607467434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:16.363244001 +0000 UTC m=+554.911662121" watchObservedRunningTime="2026-07-22 01:37:16.365347623 +0000 UTC m=+554.913765783"
Jul 22 01:37:22.380793 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:22.380763    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-fjdlg"
Jul 22 01:37:22.960059 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:22.960028    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:22.963327 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:22.963306    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:22.965697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:22.965677    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cwvnk\""
Jul 22 01:37:22.973024 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:22.973006    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:23.011383 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.011350    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t6wz\" (UniqueName: \"kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz\") pod \"authorino-5d4b67c446-7zlzn\" (UID: \"e42413b3-03aa-4a8b-9313-c81984fe32c2\") " pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:23.111762 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.111719    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t6wz\" (UniqueName: \"kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz\") pod \"authorino-5d4b67c446-7zlzn\" (UID: \"e42413b3-03aa-4a8b-9313-c81984fe32c2\") " pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:23.122081 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.122050    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t6wz\" (UniqueName: \"kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz\") pod \"authorino-5d4b67c446-7zlzn\" (UID: \"e42413b3-03aa-4a8b-9313-c81984fe32c2\") " pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:23.232640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.232548    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:23.232824 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.232808    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:23.266073 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.266033    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:37:23.274104 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.274068    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.276675 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.276649    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 01:37:23.279625 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.279598    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:37:23.364297 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.364263    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:23.367167 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:37:23.367142    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode42413b3_03aa_4a8b_9313_c81984fe32c2.slice/crio-03e2276186965e65457199553408ceccd18c5f1cf6f55477792c8d44ab4875ba WatchSource:0}: Error finding container 03e2276186965e65457199553408ceccd18c5f1cf6f55477792c8d44ab4875ba: Status 404 returned error can't find the container with id 03e2276186965e65457199553408ceccd18c5f1cf6f55477792c8d44ab4875ba
Jul 22 01:37:23.378279 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.378251    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" event={"ID":"e42413b3-03aa-4a8b-9313-c81984fe32c2","Type":"ContainerStarted","Data":"03e2276186965e65457199553408ceccd18c5f1cf6f55477792c8d44ab4875ba"}
Jul 22 01:37:23.414578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.414537    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z89v6\" (UniqueName: \"kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.414932 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.414586    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.515988 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.515882    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.516145 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.516002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z89v6\" (UniqueName: \"kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.518270 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.518250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.525550 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.525524    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z89v6\" (UniqueName: \"kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6\") pod \"authorino-64987d645d-4s9cd\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") " pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.586084 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.586055    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:37:23.723838 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:23.723779    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:37:23.729441 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:37:23.729390    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7062df1b_7237_4d58_b59b_d7a0b71f4f86.slice/crio-43ae68249d619cccc7bab338d71d91d7d5e72451990e25a56bbba13f94836e78 WatchSource:0}: Error finding container 43ae68249d619cccc7bab338d71d91d7d5e72451990e25a56bbba13f94836e78: Status 404 returned error can't find the container with id 43ae68249d619cccc7bab338d71d91d7d5e72451990e25a56bbba13f94836e78
Jul 22 01:37:24.384193 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.384095    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" podUID="e42413b3-03aa-4a8b-9313-c81984fe32c2" containerName="authorino" containerID="cri-o://ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07" gracePeriod=30
Jul 22 01:37:24.384193 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.384098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" event={"ID":"e42413b3-03aa-4a8b-9313-c81984fe32c2","Type":"ContainerStarted","Data":"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"}
Jul 22 01:37:24.385551 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.385529    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64987d645d-4s9cd" event={"ID":"7062df1b-7237-4d58-b59b-d7a0b71f4f86","Type":"ContainerStarted","Data":"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"}
Jul 22 01:37:24.385665 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.385558    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64987d645d-4s9cd" event={"ID":"7062df1b-7237-4d58-b59b-d7a0b71f4f86","Type":"ContainerStarted","Data":"43ae68249d619cccc7bab338d71d91d7d5e72451990e25a56bbba13f94836e78"}
Jul 22 01:37:24.416624 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.416570    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" podStartSLOduration=2.080360823 podStartE2EDuration="2.416550627s" totalImagesPullingTime="336.189804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:24.416149449 +0000 UTC m=+562.964567571" watchObservedRunningTime="2026-07-22 01:37:24.416550627 +0000 UTC m=+562.964968753"
Jul 22 01:37:24.454303 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.454231    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64987d645d-4s9cd" podStartSLOduration=1.06053645 podStartE2EDuration="1.454210538s" totalImagesPullingTime="393.674088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:37:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:37:24.452242651 +0000 UTC m=+563.000660774" watchObservedRunningTime="2026-07-22 01:37:24.454210538 +0000 UTC m=+563.002628660"
Jul 22 01:37:24.635634 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.635580    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:24.726274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.726236    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t6wz\" (UniqueName: \"kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz\") pod \"e42413b3-03aa-4a8b-9313-c81984fe32c2\" (UID: \"e42413b3-03aa-4a8b-9313-c81984fe32c2\") "
Jul 22 01:37:24.728213 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.728189    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz" (OuterVolumeSpecName: "kube-api-access-2t6wz") pod "e42413b3-03aa-4a8b-9313-c81984fe32c2" (UID: "e42413b3-03aa-4a8b-9313-c81984fe32c2"). InnerVolumeSpecName "kube-api-access-2t6wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:37:24.826999 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:24.826967    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t6wz\" (UniqueName: \"kubernetes.io/projected/e42413b3-03aa-4a8b-9313-c81984fe32c2-kube-api-access-2t6wz\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:37:25.390870 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.390778    2560 generic.go:356] "Generic (PLEG): container finished" podID="e42413b3-03aa-4a8b-9313-c81984fe32c2" containerID="ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07" exitCode=0
Jul 22 01:37:25.390870 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.390818    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-7zlzn"
Jul 22 01:37:25.391097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.390866    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" event={"ID":"e42413b3-03aa-4a8b-9313-c81984fe32c2","Type":"ContainerDied","Data":"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"}
Jul 22 01:37:25.391097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.390902    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-7zlzn" event={"ID":"e42413b3-03aa-4a8b-9313-c81984fe32c2","Type":"ContainerDied","Data":"03e2276186965e65457199553408ceccd18c5f1cf6f55477792c8d44ab4875ba"}
Jul 22 01:37:25.391097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.390921    2560 scope.go:117] "RemoveContainer" containerID="ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"
Jul 22 01:37:25.398778 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.398755    2560 scope.go:117] "RemoveContainer" containerID="ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"
Jul 22 01:37:25.399015 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:37:25.398995    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07\": container with ID starting with ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07 not found: ID does not exist" containerID="ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"
Jul 22 01:37:25.399077 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.399028    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07"} err="failed to get container status \"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07\": rpc error: code = NotFound desc = could not find container \"ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07\": container with ID starting with ac0e2c5bf8160349123aff01ae88835d2b07c074006fef94b65dbb9ae72f9a07 not found: ID does not exist"
Jul 22 01:37:25.431594 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.431557    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:25.448320 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.448291    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-7zlzn"]
Jul 22 01:37:25.990618 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:37:25.990545    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e42413b3-03aa-4a8b-9313-c81984fe32c2" path="/var/lib/kubelet/pods/e42413b3-03aa-4a8b-9313-c81984fe32c2/volumes"
Jul 22 01:38:06.694223 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.694187    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-bxr8b"]
Jul 22 01:38:06.694705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.694495    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e42413b3-03aa-4a8b-9313-c81984fe32c2" containerName="authorino"
Jul 22 01:38:06.694705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.694506    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e42413b3-03aa-4a8b-9313-c81984fe32c2" containerName="authorino"
Jul 22 01:38:06.694705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.694569    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e42413b3-03aa-4a8b-9313-c81984fe32c2" containerName="authorino"
Jul 22 01:38:06.698773 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.698753    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.701568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.701545    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 01:38:06.701688 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.701585    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tvqr4\""
Jul 22 01:38:06.711875 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.711851    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-bxr8b"]
Jul 22 01:38:06.779353 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.779315    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkgzx\" (UniqueName: \"kubernetes.io/projected/b467903e-6246-41df-81ea-9dec308b66fb-kube-api-access-xkgzx\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.779353 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.779353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b467903e-6246-41df-81ea-9dec308b66fb-plugins-config-volume\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.880725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.880685    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkgzx\" (UniqueName: \"kubernetes.io/projected/b467903e-6246-41df-81ea-9dec308b66fb-kube-api-access-xkgzx\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.880725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.880722    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b467903e-6246-41df-81ea-9dec308b66fb-plugins-config-volume\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.881354 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.881334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b467903e-6246-41df-81ea-9dec308b66fb-plugins-config-volume\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:06.890357 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:06.890334    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkgzx\" (UniqueName: \"kubernetes.io/projected/b467903e-6246-41df-81ea-9dec308b66fb-kube-api-access-xkgzx\") pod \"payload-processing-5bf47cd988-bxr8b\" (UID: \"b467903e-6246-41df-81ea-9dec308b66fb\") " pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:07.009033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:07.008952    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:07.147358 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:07.147333    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-bxr8b"]
Jul 22 01:38:07.150005 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:38:07.149974    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb467903e_6246_41df_81ea_9dec308b66fb.slice/crio-4009fabef4feb718d3b084f44d60711eec60030f09e4dbd5b778fc7e154920d6 WatchSource:0}: Error finding container 4009fabef4feb718d3b084f44d60711eec60030f09e4dbd5b778fc7e154920d6: Status 404 returned error can't find the container with id 4009fabef4feb718d3b084f44d60711eec60030f09e4dbd5b778fc7e154920d6
Jul 22 01:38:07.571127 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:07.571094    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b" event={"ID":"b467903e-6246-41df-81ea-9dec308b66fb","Type":"ContainerStarted","Data":"4009fabef4feb718d3b084f44d60711eec60030f09e4dbd5b778fc7e154920d6"}
Jul 22 01:38:10.588487 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:10.588448    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b" event={"ID":"b467903e-6246-41df-81ea-9dec308b66fb","Type":"ContainerStarted","Data":"fd8e4d993a1c62447262446536ff5812e767a2b0038d996debd92873271385b5"}
Jul 22 01:38:10.588911 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:10.588518    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:10.617820 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:10.617773    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b" podStartSLOduration=1.876425505 podStartE2EDuration="4.617756017s" totalImagesPullingTime="2.741330512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:10.617442665 +0000 UTC m=+609.165860812" watchObservedRunningTime="2026-07-22 01:38:10.617756017 +0000 UTC m=+609.166174141"
Jul 22 01:38:21.594182 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:21.594152    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-bxr8b"
Jul 22 01:38:26.802120 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.802040    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"]
Jul 22 01:38:26.805161 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.805145    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.810850 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.810824    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sk8d9\""
Jul 22 01:38:26.810978 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.810859    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 01:38:26.810978 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.810826    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 22 01:38:26.810978 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.810833    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 01:38:26.820668 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.820645    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"]
Jul 22 01:38:26.957841 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.957809    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25478eee-88d1-4d56-b9b6-0b082d411c26-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.958012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.957859    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsp8h\" (UniqueName: \"kubernetes.io/projected/25478eee-88d1-4d56-b9b6-0b082d411c26-kube-api-access-dsp8h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.958012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.957883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.958012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.957909    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.958012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.957931    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:26.958169 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:26.958017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058502 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058418    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058502 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058474    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25478eee-88d1-4d56-b9b6-0b082d411c26-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058502 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058505    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsp8h\" (UniqueName: \"kubernetes.io/projected/25478eee-88d1-4d56-b9b6-0b082d411c26-kube-api-access-dsp8h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058766 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058534    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058766 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058569    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058766 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058605    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.058977 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058950    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.059041 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.058985    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.059077 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.059051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.060725 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.060703    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25478eee-88d1-4d56-b9b6-0b082d411c26-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.060957 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.060941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25478eee-88d1-4d56-b9b6-0b082d411c26-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.092452 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.092425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsp8h\" (UniqueName: \"kubernetes.io/projected/25478eee-88d1-4d56-b9b6-0b082d411c26-kube-api-access-dsp8h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv\" (UID: \"25478eee-88d1-4d56-b9b6-0b082d411c26\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.114487 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.114461    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:27.245155 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.245038    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"]
Jul 22 01:38:27.247744 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:38:27.247717    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25478eee_88d1_4d56_b9b6_0b082d411c26.slice/crio-ca0dec4b7bd7a76ecb980dfead4645ab0553b655e41493a5bb46c7199d29d1d6 WatchSource:0}: Error finding container ca0dec4b7bd7a76ecb980dfead4645ab0553b655e41493a5bb46c7199d29d1d6: Status 404 returned error can't find the container with id ca0dec4b7bd7a76ecb980dfead4645ab0553b655e41493a5bb46c7199d29d1d6
Jul 22 01:38:27.665826 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:27.665775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv" event={"ID":"25478eee-88d1-4d56-b9b6-0b082d411c26","Type":"ContainerStarted","Data":"ca0dec4b7bd7a76ecb980dfead4645ab0553b655e41493a5bb46c7199d29d1d6"}
Jul 22 01:38:28.671515 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:28.671478    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv" event={"ID":"25478eee-88d1-4d56-b9b6-0b082d411c26","Type":"ContainerStarted","Data":"7020b7a4b843d9b275236709b2f6bcb1fe6fa269e7e252dcd4a3c411547719db"}
Jul 22 01:38:28.709515 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:28.709465    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv" podStartSLOduration=1.484966823 podStartE2EDuration="2.709448669s" totalImagesPullingTime="1.224481846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:28.707951462 +0000 UTC m=+627.256369631" watchObservedRunningTime="2026-07-22 01:38:28.709448669 +0000 UTC m=+627.257866792"
Jul 22 01:38:37.115266 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:37.115223    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:37.115764 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:37.115300    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:37.127245 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:37.127220    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:37.719677 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:37.719644    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv"
Jul 22 01:38:53.038934 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.038901    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"]
Jul 22 01:38:53.042037 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.042022    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.047762 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.047732    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 01:38:53.077914 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.077886    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"]
Jul 22 01:38:53.091879 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.091846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.091879 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.091880    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f83dfe2-b843-41b9-ba57-50d268932db4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.092082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.091959    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.092082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.091998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.092082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.092017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh6gk\" (UniqueName: \"kubernetes.io/projected/0f83dfe2-b843-41b9-ba57-50d268932db4-kube-api-access-bh6gk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.092082 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.092046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192656 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192616    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh6gk\" (UniqueName: \"kubernetes.io/projected/0f83dfe2-b843-41b9-ba57-50d268932db4-kube-api-access-bh6gk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192656 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192664    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192920 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192720    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192920 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f83dfe2-b843-41b9-ba57-50d268932db4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192920 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.192920 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.192786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.193178 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.193150    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.193307 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.193221    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.193390 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.193368    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.194957 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.194939    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f83dfe2-b843-41b9-ba57-50d268932db4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.195207 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.195190    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f83dfe2-b843-41b9-ba57-50d268932db4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.201994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.201967    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh6gk\" (UniqueName: \"kubernetes.io/projected/0f83dfe2-b843-41b9-ba57-50d268932db4-kube-api-access-bh6gk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vb74w\" (UID: \"0f83dfe2-b843-41b9-ba57-50d268932db4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.351578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.351489    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:38:53.484097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.484065    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"]
Jul 22 01:38:53.487815 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:38:53.487777    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f83dfe2_b843_41b9_ba57_50d268932db4.slice/crio-3d9c115bc6a8cdd9841f7de81d62cba0c737a2695824a2b15097d46ab48e40ef WatchSource:0}: Error finding container 3d9c115bc6a8cdd9841f7de81d62cba0c737a2695824a2b15097d46ab48e40ef: Status 404 returned error can't find the container with id 3d9c115bc6a8cdd9841f7de81d62cba0c737a2695824a2b15097d46ab48e40ef
Jul 22 01:38:53.491825 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.491805    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 01:38:53.789648 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.789617    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w" event={"ID":"0f83dfe2-b843-41b9-ba57-50d268932db4","Type":"ContainerStarted","Data":"e4aa68bceda3f0cc619212a6422fbd2dcd4cfcc219c5ec10aa878789f10806fe"}
Jul 22 01:38:53.789796 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:38:53.789668    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w" event={"ID":"0f83dfe2-b843-41b9-ba57-50d268932db4","Type":"ContainerStarted","Data":"3d9c115bc6a8cdd9841f7de81d62cba0c737a2695824a2b15097d46ab48e40ef"}
Jul 22 01:39:03.352134 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:03.352093    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:39:03.352561 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:03.352195    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:39:03.364410 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:03.364368    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:39:03.388667 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:03.388618    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w" podStartSLOduration=10.211561965 podStartE2EDuration="10.388602696s" totalImagesPullingTime="177.040731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:38:53.814029228 +0000 UTC m=+652.362447353" watchObservedRunningTime="2026-07-22 01:39:03.388602696 +0000 UTC m=+661.937020849"
Jul 22 01:39:03.849956 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:03.849925    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vb74w"
Jul 22 01:39:51.765530 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:51.765497    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:39:51.766069 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:51.765665    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-64987d645d-4s9cd" podUID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" containerName="authorino" containerID="cri-o://880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754" gracePeriod=30
Jul 22 01:39:52.011054 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.011025    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:39:52.047494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.047369    2560 generic.go:356] "Generic (PLEG): container finished" podID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" containerID="880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754" exitCode=0
Jul 22 01:39:52.047494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.047432    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64987d645d-4s9cd"
Jul 22 01:39:52.047494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.047432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64987d645d-4s9cd" event={"ID":"7062df1b-7237-4d58-b59b-d7a0b71f4f86","Type":"ContainerDied","Data":"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"}
Jul 22 01:39:52.047494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.047480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64987d645d-4s9cd" event={"ID":"7062df1b-7237-4d58-b59b-d7a0b71f4f86","Type":"ContainerDied","Data":"43ae68249d619cccc7bab338d71d91d7d5e72451990e25a56bbba13f94836e78"}
Jul 22 01:39:52.047750 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.047503    2560 scope.go:117] "RemoveContainer" containerID="880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"
Jul 22 01:39:52.055387 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.055370    2560 scope.go:117] "RemoveContainer" containerID="880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"
Jul 22 01:39:52.055706 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:39:52.055675    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754\": container with ID starting with 880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754 not found: ID does not exist" containerID="880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"
Jul 22 01:39:52.055805 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.055707    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754"} err="failed to get container status \"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754\": rpc error: code = NotFound desc = could not find container \"880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754\": container with ID starting with 880915c9caf351eece4a330444c09fc696f17a2a457d4d96944d106183f08754 not found: ID does not exist"
Jul 22 01:39:52.126138 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.126098    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z89v6\" (UniqueName: \"kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6\") pod \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") "
Jul 22 01:39:52.126321 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.126183    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert\") pod \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\" (UID: \"7062df1b-7237-4d58-b59b-d7a0b71f4f86\") "
Jul 22 01:39:52.128143 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.128111    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6" (OuterVolumeSpecName: "kube-api-access-z89v6") pod "7062df1b-7237-4d58-b59b-d7a0b71f4f86" (UID: "7062df1b-7237-4d58-b59b-d7a0b71f4f86"). InnerVolumeSpecName "kube-api-access-z89v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:39:52.135143 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.135121    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7062df1b-7237-4d58-b59b-d7a0b71f4f86" (UID: "7062df1b-7237-4d58-b59b-d7a0b71f4f86"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:39:52.227486 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.227442    2560 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7062df1b-7237-4d58-b59b-d7a0b71f4f86-tls-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:39:52.227486 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.227477    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z89v6\" (UniqueName: \"kubernetes.io/projected/7062df1b-7237-4d58-b59b-d7a0b71f4f86-kube-api-access-z89v6\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:39:52.368856 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.368827    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:39:52.371052 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:52.371027    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-64987d645d-4s9cd"]
Jul 22 01:39:53.990119 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:39:53.990039    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" path="/var/lib/kubelet/pods/7062df1b-7237-4d58-b59b-d7a0b71f4f86/volumes"
Jul 22 01:45:00.147537 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.147441    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"]
Jul 22 01:45:00.149929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.147930    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" containerName="authorino"
Jul 22 01:45:00.149929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.147948    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" containerName="authorino"
Jul 22 01:45:00.149929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.148032    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7062df1b-7237-4d58-b59b-d7a0b71f4f86" containerName="authorino"
Jul 22 01:45:00.150929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.150909    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"
Jul 22 01:45:00.153583 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.153565    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:45:00.160333 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.160316    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"
Jul 22 01:45:00.168709 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.168685    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"]
Jul 22 01:45:00.296184 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.296153    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"]
Jul 22 01:45:00.298736 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:45:00.298706    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c11ab0_6bb7_479e_acde_c59d52e492bf.slice/crio-97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d WatchSource:0}: Error finding container 97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d: Status 404 returned error can't find the container with id 97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d
Jul 22 01:45:00.300609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.300592    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 01:45:00.383193 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:00.383152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w" event={"ID":"a0c11ab0-6bb7-479e-acde-c59d52e492bf","Type":"ContainerStarted","Data":"97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d"}
Jul 22 01:45:04.402765 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:04.402732    2560 generic.go:356] "Generic (PLEG): container finished" podID="a0c11ab0-6bb7-479e-acde-c59d52e492bf" containerID="646fdafd983a8e37d009982a1609e048c1f697ff6c8c4f421e1b39ba51f38c62" exitCode=0
Jul 22 01:45:04.403134 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:04.402784    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w" event={"ID":"a0c11ab0-6bb7-479e-acde-c59d52e492bf","Type":"ContainerDied","Data":"646fdafd983a8e37d009982a1609e048c1f697ff6c8c4f421e1b39ba51f38c62"}
Jul 22 01:45:05.534563 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:05.534532    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"
Jul 22 01:45:06.413273 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:06.413238    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w" event={"ID":"a0c11ab0-6bb7-479e-acde-c59d52e492bf","Type":"ContainerDied","Data":"97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d"}
Jul 22 01:45:06.413455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:06.413293    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97c907fe4be648939a4e1f88b83ba9ac50ed9e7edd02a7aa21998b60b7ec1f0d"
Jul 22 01:45:06.413455 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:45:06.413248    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744745-2mh6w"
Jul 22 01:50:32.125528 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.125452    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:50:32.127657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.125768    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c11ab0-6bb7-479e-acde-c59d52e492bf" containerName="cleanup"
Jul 22 01:50:32.127657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.125778    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c11ab0-6bb7-479e-acde-c59d52e492bf" containerName="cleanup"
Jul 22 01:50:32.127657 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.125843    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c11ab0-6bb7-479e-acde-c59d52e492bf" containerName="cleanup"
Jul 22 01:50:32.128568 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.128551    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.131092 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.131062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-6509edbf-kserve-self-signed-certs\""
Jul 22 01:50:32.146767 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.146738    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:50:32.183796 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183762    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.183953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.183953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183855    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.183953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhj82\" (UniqueName: \"kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.183953 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.184099 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.183969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285290 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285258    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285298    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhj82\" (UniqueName: \"kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285460    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285501    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.285852 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.285827    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.286164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.286113    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.286164 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.286136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.287578 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.287558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.287960 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.287942    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.292770 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.292749    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhj82\" (UniqueName: \"kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82\") pod \"e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") " pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.437807 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.437721    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:32.563662 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.563638    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:50:32.566118 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:50:32.566088    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f82d7f_ad4e_4774_a173_0b408c6a9de7.slice/crio-2c750dbc3c24076fb315a7e676bf65c9a8c79d6b8f9ea3fac5e6ccee35b2941d WatchSource:0}: Error finding container 2c750dbc3c24076fb315a7e676bf65c9a8c79d6b8f9ea3fac5e6ccee35b2941d: Status 404 returned error can't find the container with id 2c750dbc3c24076fb315a7e676bf65c9a8c79d6b8f9ea3fac5e6ccee35b2941d
Jul 22 01:50:32.830554 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.830465    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" event={"ID":"20f82d7f-ad4e-4774-a173-0b408c6a9de7","Type":"ContainerStarted","Data":"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"}
Jul 22 01:50:32.830554 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.830511    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" event={"ID":"20f82d7f-ad4e-4774-a173-0b408c6a9de7","Type":"ContainerStarted","Data":"2c750dbc3c24076fb315a7e676bf65c9a8c79d6b8f9ea3fac5e6ccee35b2941d"}
Jul 22 01:50:32.848933 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:32.848886    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" podStartSLOduration=0.848869529 podStartE2EDuration="848.869529ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:50:32.847510035 +0000 UTC m=+1351.395928161" watchObservedRunningTime="2026-07-22 01:50:32.848869529 +0000 UTC m=+1351.397287652"
Jul 22 01:50:41.818456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.818419    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:50:41.822300 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.822277    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.824644 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.824618    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-6509edbf-kserve-self-signed-certs\""
Jul 22 01:50:41.829869 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.829843    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:50:41.973439 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.973439 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973443    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.973697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973470    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5dcw\" (UniqueName: \"kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.973697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973510    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.973697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973576    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:41.973697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:41.973626    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074280 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074181    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074280 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074226    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074547 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074323    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5dcw\" (UniqueName: \"kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074547 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074364    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074547 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074429    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074547 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074827 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.074875 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.075007 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.074991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.076603 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.076573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.076791 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.076771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.082191 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.082168    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5dcw\" (UniqueName: \"kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw\") pod \"e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") " pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.133936 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.133898    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:42.267547 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.267529    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:50:42.270291 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:50:42.270264    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cd3719a_1a09_495c_aff3_7d45329b13ea.slice/crio-63c89470a0215866cb3f417b083b85e9b8fd540aa3a57cc224d11f1af0aede5e WatchSource:0}: Error finding container 63c89470a0215866cb3f417b083b85e9b8fd540aa3a57cc224d11f1af0aede5e: Status 404 returned error can't find the container with id 63c89470a0215866cb3f417b083b85e9b8fd540aa3a57cc224d11f1af0aede5e
Jul 22 01:50:42.438853 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.438816    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:42.438853 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.438863    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:42.451365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.451331    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:42.878130 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.878098    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" event={"ID":"4cd3719a-1a09-495c-aff3-7d45329b13ea","Type":"ContainerStarted","Data":"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"}
Jul 22 01:50:42.878530 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.878139    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" event={"ID":"4cd3719a-1a09-495c-aff3-7d45329b13ea","Type":"ContainerStarted","Data":"63c89470a0215866cb3f417b083b85e9b8fd540aa3a57cc224d11f1af0aede5e"}
Jul 22 01:50:42.889696 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.889669    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:50:42.894828 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:42.894781    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" podStartSLOduration=1.8947642120000001 podStartE2EDuration="1.894764212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:50:42.894243758 +0000 UTC m=+1361.442661882" watchObservedRunningTime="2026-07-22 01:50:42.894764212 +0000 UTC m=+1361.443182335"
Jul 22 01:50:52.134493 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:52.134460    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:52.135004 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:52.134505    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:52.147418 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:52.147370    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:50:52.939887 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:50:52.939860    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:51:01.274789 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:01.274756    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:51:02.258613 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:02.258567    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:51:16.294769 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.294700    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" podUID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" containerName="main" containerID="cri-o://10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e" gracePeriod=45
Jul 22 01:51:16.545867 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.545807    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:51:16.576815 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.576790    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.576958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.576836    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.576958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.576860    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhj82\" (UniqueName: \"kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.576958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.576891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.576958 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.576921    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.577154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577006    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache\") pod \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\" (UID: \"20f82d7f-ad4e-4774-a173-0b408c6a9de7\") "
Jul 22 01:51:16.577154 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577120    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:16.577316 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577159    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home" (OuterVolumeSpecName: "home") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:16.577368 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577317    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:16.577368 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577331    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:16.577368 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.577326    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache" (OuterVolumeSpecName: "model-cache") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:16.579144 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.579116    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm" (OuterVolumeSpecName: "dshm") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:16.579460 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.579438    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:51:16.579534 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.579440    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82" (OuterVolumeSpecName: "kube-api-access-qhj82") pod "20f82d7f-ad4e-4774-a173-0b408c6a9de7" (UID: "20f82d7f-ad4e-4774-a173-0b408c6a9de7"). InnerVolumeSpecName "kube-api-access-qhj82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:51:16.678553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.678505    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f82d7f-ad4e-4774-a173-0b408c6a9de7-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:16.678553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.678547    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhj82\" (UniqueName: \"kubernetes.io/projected/20f82d7f-ad4e-4774-a173-0b408c6a9de7-kube-api-access-qhj82\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:16.678553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.678557    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:16.678553 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:16.678567    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f82d7f-ad4e-4774-a173-0b408c6a9de7-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.037982 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.037951    2560 generic.go:356] "Generic (PLEG): container finished" podID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" containerID="10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e" exitCode=0
Jul 22 01:51:17.038151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.037990    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"
Jul 22 01:51:17.038151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.038037    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" event={"ID":"20f82d7f-ad4e-4774-a173-0b408c6a9de7","Type":"ContainerDied","Data":"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"}
Jul 22 01:51:17.038151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.038077    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk" event={"ID":"20f82d7f-ad4e-4774-a173-0b408c6a9de7","Type":"ContainerDied","Data":"2c750dbc3c24076fb315a7e676bf65c9a8c79d6b8f9ea3fac5e6ccee35b2941d"}
Jul 22 01:51:17.038151 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.038096    2560 scope.go:117] "RemoveContainer" containerID="10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"
Jul 22 01:51:17.047043 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.047026    2560 scope.go:117] "RemoveContainer" containerID="10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"
Jul 22 01:51:17.047328 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:51:17.047308    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e\": container with ID starting with 10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e not found: ID does not exist" containerID="10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"
Jul 22 01:51:17.047471 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.047336    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e"} err="failed to get container status \"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e\": rpc error: code = NotFound desc = could not find container \"10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e\": container with ID starting with 10f3410390d42aba878d7c983ca31e7ad708de0facbd4aec49e737e46f12556e not found: ID does not exist"
Jul 22 01:51:17.065591 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.065565    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:51:17.069210 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.069188    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-6509edbf-kserve-79f967475f-pswmk"]
Jul 22 01:51:17.279671 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.279615    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" podUID="4cd3719a-1a09-495c-aff3-7d45329b13ea" containerName="main" containerID="cri-o://e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a" gracePeriod=45
Jul 22 01:51:17.537259 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.537232    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:51:17.586424 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586336    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586424 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586378    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5dcw\" (UniqueName: \"kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586442    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586467    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586522    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir\") pod \"4cd3719a-1a09-495c-aff3-7d45329b13ea\" (UID: \"4cd3719a-1a09-495c-aff3-7d45329b13ea\") "
Jul 22 01:51:17.586862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586728    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:17.586862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586744    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home" (OuterVolumeSpecName: "home") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:17.586862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586803    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.586862 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586818    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.587062 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.586923    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:17.588705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.588681    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 01:51:17.588817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.588767    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm" (OuterVolumeSpecName: "dshm") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:51:17.588817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.588783    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw" (OuterVolumeSpecName: "kube-api-access-b5dcw") pod "4cd3719a-1a09-495c-aff3-7d45329b13ea" (UID: "4cd3719a-1a09-495c-aff3-7d45329b13ea"). InnerVolumeSpecName "kube-api-access-b5dcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:51:17.687623 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.687585    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.687623 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.687617    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd3719a-1a09-495c-aff3-7d45329b13ea-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.687623 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.687627    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cd3719a-1a09-495c-aff3-7d45329b13ea-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.687857 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.687635    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5dcw\" (UniqueName: \"kubernetes.io/projected/4cd3719a-1a09-495c-aff3-7d45329b13ea-kube-api-access-b5dcw\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:51:17.991345 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:17.991311    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" path="/var/lib/kubelet/pods/20f82d7f-ad4e-4774-a173-0b408c6a9de7/volumes"
Jul 22 01:51:18.044559 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.044524    2560 generic.go:356] "Generic (PLEG): container finished" podID="4cd3719a-1a09-495c-aff3-7d45329b13ea" containerID="e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a" exitCode=0
Jul 22 01:51:18.044721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.044566    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"
Jul 22 01:51:18.044721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.044611    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" event={"ID":"4cd3719a-1a09-495c-aff3-7d45329b13ea","Type":"ContainerDied","Data":"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"}
Jul 22 01:51:18.044721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.044664    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k" event={"ID":"4cd3719a-1a09-495c-aff3-7d45329b13ea","Type":"ContainerDied","Data":"63c89470a0215866cb3f417b083b85e9b8fd540aa3a57cc224d11f1af0aede5e"}
Jul 22 01:51:18.044721 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.044688    2560 scope.go:117] "RemoveContainer" containerID="e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"
Jul 22 01:51:18.052881 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.052860    2560 scope.go:117] "RemoveContainer" containerID="e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"
Jul 22 01:51:18.053092 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:51:18.053077    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a\": container with ID starting with e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a not found: ID does not exist" containerID="e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"
Jul 22 01:51:18.053171 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.053097    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a"} err="failed to get container status \"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a\": rpc error: code = NotFound desc = could not find container \"e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a\": container with ID starting with e6ae4b14ec010a2e0726904bb9f02549d52a61c75b512ab1e99192f9f162857a not found: ID does not exist"
Jul 22 01:51:18.062586 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.062557    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:51:18.065161 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:18.065142    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-6509edbf-kserve-d5f5d9fdc-ldl8k"]
Jul 22 01:51:19.990227 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:51:19.990195    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cd3719a-1a09-495c-aff3-7d45329b13ea" path="/var/lib/kubelet/pods/4cd3719a-1a09-495c-aff3-7d45329b13ea/volumes"
Jul 22 01:55:13.784698 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.784663    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"]
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.784981    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" containerName="main"
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.784992    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" containerName="main"
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.785012    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cd3719a-1a09-495c-aff3-7d45329b13ea" containerName="main"
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.785017    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cd3719a-1a09-495c-aff3-7d45329b13ea" containerName="main"
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.785072    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f82d7f-ad4e-4774-a173-0b408c6a9de7" containerName="main"
Jul 22 01:55:13.785111 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.785082    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cd3719a-1a09-495c-aff3-7d45329b13ea" containerName="main"
Jul 22 01:55:13.788097 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.788080    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:13.790511 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.790491    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:55:13.796232 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.796202    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"]
Jul 22 01:55:13.892351 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.892309    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4\" (UID: \"08224216-9046-41f2-b3ca-ee2ba67171bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:13.993360 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.993326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4\" (UID: \"08224216-9046-41f2-b3ca-ee2ba67171bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:13.993890 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:13.993871    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4\" (UID: \"08224216-9046-41f2-b3ca-ee2ba67171bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:14.098981 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:14.098896    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:14.234798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:14.234773    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"]
Jul 22 01:55:14.236854 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:55:14.236829    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08224216_9046_41f2_b3ca_ee2ba67171bd.slice/crio-3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe WatchSource:0}: Error finding container 3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe: Status 404 returned error can't find the container with id 3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe
Jul 22 01:55:14.881712 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:55:14.881680    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08224216_9046_41f2_b3ca_ee2ba67171bd.slice/crio-conmon-44c7c459bf7d566b30464e5c878b321291d5143cb4b4d59dc30180fd124f257f.scope\": RecentStats: unable to find data in memory cache]"
Jul 22 01:55:15.108200 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:15.108163    2560 generic.go:356] "Generic (PLEG): container finished" podID="08224216-9046-41f2-b3ca-ee2ba67171bd" containerID="44c7c459bf7d566b30464e5c878b321291d5143cb4b4d59dc30180fd124f257f" exitCode=0
Jul 22 01:55:15.108369 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:15.108226    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4" event={"ID":"08224216-9046-41f2-b3ca-ee2ba67171bd","Type":"ContainerDied","Data":"44c7c459bf7d566b30464e5c878b321291d5143cb4b4d59dc30180fd124f257f"}
Jul 22 01:55:15.108369 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:15.108284    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4" event={"ID":"08224216-9046-41f2-b3ca-ee2ba67171bd","Type":"ContainerStarted","Data":"3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe"}
Jul 22 01:55:16.244160 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:16.244136    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:16.313865 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:16.313828    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca\") pod \"08224216-9046-41f2-b3ca-ee2ba67171bd\" (UID: \"08224216-9046-41f2-b3ca-ee2ba67171bd\") "
Jul 22 01:55:16.314177 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:16.314153    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08224216-9046-41f2-b3ca-ee2ba67171bd" (UID: "08224216-9046-41f2-b3ca-ee2ba67171bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:55:16.414497 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:16.414414    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08224216-9046-41f2-b3ca-ee2ba67171bd-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:55:17.122846 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:17.122805    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4" event={"ID":"08224216-9046-41f2-b3ca-ee2ba67171bd","Type":"ContainerDied","Data":"3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe"}
Jul 22 01:55:17.123047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:17.122901    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ccf1de401d102af3bcd8fb0cf2b919647d2bdeea5fc5581e42db8c344a39efe"
Jul 22 01:55:17.123047 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:17.123026    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"
Jul 22 01:55:28.858531 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:28.858497    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"]
Jul 22 01:55:28.860304 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:28.860263    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1afc4013-pbzx4"]
Jul 22 01:55:29.991202 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:29.991165    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08224216-9046-41f2-b3ca-ee2ba67171bd" path="/var/lib/kubelet/pods/08224216-9046-41f2-b3ca-ee2ba67171bd/volumes"
Jul 22 01:55:30.077890 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.077862    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:55:30.078381 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.078357    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08224216-9046-41f2-b3ca-ee2ba67171bd" containerName="revoke-keys"
Jul 22 01:55:30.078381 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.078377    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="08224216-9046-41f2-b3ca-ee2ba67171bd" containerName="revoke-keys"
Jul 22 01:55:30.078481 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.078463    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="08224216-9046-41f2-b3ca-ee2ba67171bd" containerName="revoke-keys"
Jul 22 01:55:30.081582 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.081563    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.084057 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.084037    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 01:55:30.084219 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.084196    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fffbfad3-gw-openshift-default-dockercfg-zcscl\""
Jul 22 01:55:30.092309 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.092286    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:55:30.238478 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238478 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238713 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238572    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238713 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238611    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238713 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vtk6\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238713 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238869 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238715    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238869 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238742    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.238869 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.238810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340099 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340016    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vtk6\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340099 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340057    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340099 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340088    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340231    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340685 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340685 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340536    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340685 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340627    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.340880 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.340858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.341038 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.341021    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.342475 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.342455    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.342798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.342778    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.348145 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.348120    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.348335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.348314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vtk6\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6\") pod \"e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") " pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.392543 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.392517    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:30.529637 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.529604    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:55:30.532577 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:55:30.532548    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54d24bc_343d_4360_894d_c43b9cf63b20.slice/crio-acc15d1b23803ee17b56da4b6c0e21c85ae79cb1b5aa8278287d7e9cf5f9927b WatchSource:0}: Error finding container acc15d1b23803ee17b56da4b6c0e21c85ae79cb1b5aa8278287d7e9cf5f9927b: Status 404 returned error can't find the container with id acc15d1b23803ee17b56da4b6c0e21c85ae79cb1b5aa8278287d7e9cf5f9927b
Jul 22 01:55:30.534522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:30.534505    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 01:55:31.194446 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:31.194411    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerStarted","Data":"acc15d1b23803ee17b56da4b6c0e21c85ae79cb1b5aa8278287d7e9cf5f9927b"}
Jul 22 01:55:33.605977 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:33.605446    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:33.605977 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:33.605521    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:33.605977 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:33.605556    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:34.210560 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:34.210518    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerStarted","Data":"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"}
Jul 22 01:55:34.229938 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:34.229873    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podStartSLOduration=1.159340773 podStartE2EDuration="4.229853457s" totalImagesPullingTime="3.070512684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:55:34.22945673 +0000 UTC m=+1652.777874870" watchObservedRunningTime="2026-07-22 01:55:34.229853457 +0000 UTC m=+1652.778271582"
Jul 22 01:55:34.393244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:34.393211    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:34.397978 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:34.397955    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:35.215938 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:35.215908    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:35.216826 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:35.216808    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:47.279970 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.279938    2560 generic.go:356] "Generic (PLEG): container finished" podID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerID="37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23" exitCode=0
Jul 22 01:55:47.280371 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.280016    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerDied","Data":"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"}
Jul 22 01:55:47.280449 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.280434    2560 scope.go:117] "RemoveContainer" containerID="37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"
Jul 22 01:55:47.281436 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.281407    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:47.281542 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.281474    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:47.281542 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:47.281505    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:55:48.290066 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:48.290014    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerStarted","Data":"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"}
Jul 22 01:55:48.393489 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:48.393453    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:48.394450 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:48.394423    2560 patch_prober.go:28] interesting pod/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 22 01:55:48.394510 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:48.394482    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 22 01:55:49.393728 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:49.393689    2560 patch_prober.go:28] interesting pod/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 22 01:55:49.394118 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:49.393760    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 22 01:55:50.216779 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:50.216729    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:50.392974 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:50.392938    2560 patch_prober.go:28] interesting pod/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 22 01:55:50.393135 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:50.392997    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 22 01:55:51.393922 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:51.393880    2560 patch_prober.go:28] interesting pod/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 22 01:55:51.394436 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:51.393947    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 22 01:55:52.393495 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:52.393465    2560 patch_prober.go:28] interesting pod/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 22 01:55:52.393651 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:52.393516    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 22 01:55:53.397089 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:53.397056    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:55:53.397912 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:55:53.397891    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:56:00.839142 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:00.839105    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"]
Jul 22 01:56:00.847908 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:00.847890    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:00.851896 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:00.851861    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:56:00.854536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:00.854501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"]
Jul 22 01:56:00.927077 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:00.927035    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d\" (UID: \"582329e9-7ef7-4309-ac3c-753195267aa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:01.028407 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:01.028361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d\" (UID: \"582329e9-7ef7-4309-ac3c-753195267aa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:01.029015 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:01.028996    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d\" (UID: \"582329e9-7ef7-4309-ac3c-753195267aa6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:01.161296 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:01.161257    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:01.292609 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:01.292585    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"]
Jul 22 01:56:01.294467 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:56:01.294444    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582329e9_7ef7_4309_ac3c_753195267aa6.slice/crio-60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1 WatchSource:0}: Error finding container 60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1: Status 404 returned error can't find the container with id 60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1
Jul 22 01:56:01.349987 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:01.349953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d" event={"ID":"582329e9-7ef7-4309-ac3c-753195267aa6","Type":"ContainerStarted","Data":"60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1"}
Jul 22 01:56:02.355997 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:02.355964    2560 generic.go:356] "Generic (PLEG): container finished" podID="582329e9-7ef7-4309-ac3c-753195267aa6" containerID="2f22304de3626a54260333cf736f39af395e80fa696fd778220c5943581931fb" exitCode=0
Jul 22 01:56:02.356355 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:02.356048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d" event={"ID":"582329e9-7ef7-4309-ac3c-753195267aa6","Type":"ContainerDied","Data":"2f22304de3626a54260333cf736f39af395e80fa696fd778220c5943581931fb"}
Jul 22 01:56:03.489783 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:03.489755    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:03.549753 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:03.549719    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca\") pod \"582329e9-7ef7-4309-ac3c-753195267aa6\" (UID: \"582329e9-7ef7-4309-ac3c-753195267aa6\") "
Jul 22 01:56:03.550076 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:03.550051    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "582329e9-7ef7-4309-ac3c-753195267aa6" (UID: "582329e9-7ef7-4309-ac3c-753195267aa6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:56:03.650650 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:03.650573    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/582329e9-7ef7-4309-ac3c-753195267aa6-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:04.367371 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:04.367344    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"
Jul 22 01:56:04.367371 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:04.367361    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d" event={"ID":"582329e9-7ef7-4309-ac3c-753195267aa6","Type":"ContainerDied","Data":"60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1"}
Jul 22 01:56:04.367585 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:04.367430    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60b5370c74fde315007cf50f5fb9dffde25a2673595bc3fcfeccaa5ecd890de1"
Jul 22 01:56:15.905580 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:15.905542    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"]
Jul 22 01:56:15.908351 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:15.908330    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-fffbfad3-p2l8d"]
Jul 22 01:56:15.990817 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:15.990784    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="582329e9-7ef7-4309-ac3c-753195267aa6" path="/var/lib/kubelet/pods/582329e9-7ef7-4309-ac3c-753195267aa6/volumes"
Jul 22 01:56:16.052897 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:16.052869    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:56:16.053150 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:16.053123    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy" containerID="cri-o://b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb" gracePeriod=30
Jul 22 01:56:21.308599 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.308573    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:56:21.412871 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.412826    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vtk6\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.412871 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.412879    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413146 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.412904    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413146 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.412929    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413146 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413012    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413146 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413146 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413093    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413204    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413212    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:56:21.413433 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413268    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert\") pod \"f54d24bc-343d-4360-894d-c43b9cf63b20\" (UID: \"f54d24bc-343d-4360-894d-c43b9cf63b20\") "
Jul 22 01:56:21.413600 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413458    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:56:21.413600 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413587    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.413712 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413605    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.413834 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413807    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:56:21.413834 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413809    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:56:21.413998 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.413958    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data" (OuterVolumeSpecName: "istio-data") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:56:21.415325 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.415300    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token" (OuterVolumeSpecName: "istio-token") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:56:21.415466 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.415323    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6" (OuterVolumeSpecName: "kube-api-access-2vtk6") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "kube-api-access-2vtk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:56:21.415530 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.415507    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 01:56:21.415625 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.415612    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f54d24bc-343d-4360-894d-c43b9cf63b20" (UID: "f54d24bc-343d-4360-894d-c43b9cf63b20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:56:21.448456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.448426    2560 generic.go:356] "Generic (PLEG): container finished" podID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerID="b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb" exitCode=0
Jul 22 01:56:21.448622 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.448472    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"
Jul 22 01:56:21.448622 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.448505    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerDied","Data":"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"}
Jul 22 01:56:21.448622 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.448555    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk" event={"ID":"f54d24bc-343d-4360-894d-c43b9cf63b20","Type":"ContainerDied","Data":"acc15d1b23803ee17b56da4b6c0e21c85ae79cb1b5aa8278287d7e9cf5f9927b"}
Jul 22 01:56:21.448622 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.448576    2560 scope.go:117] "RemoveContainer" containerID="b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"
Jul 22 01:56:21.457705 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.457690    2560 scope.go:117] "RemoveContainer" containerID="37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"
Jul 22 01:56:21.465213 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.465197    2560 scope.go:117] "RemoveContainer" containerID="b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"
Jul 22 01:56:21.465473 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:56:21.465452    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb\": container with ID starting with b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb not found: ID does not exist" containerID="b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"
Jul 22 01:56:21.465536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.465482    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb"} err="failed to get container status \"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb\": rpc error: code = NotFound desc = could not find container \"b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb\": container with ID starting with b87120c02662429f9f2dbb860a81a632af0ff974d83caca487f65b1a4440a9eb not found: ID does not exist"
Jul 22 01:56:21.465536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.465498    2560 scope.go:117] "RemoveContainer" containerID="37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"
Jul 22 01:56:21.465737 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:56:21.465720    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23\": container with ID starting with 37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23 not found: ID does not exist" containerID="37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"
Jul 22 01:56:21.465779 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.465743    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23"} err="failed to get container status \"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23\": rpc error: code = NotFound desc = could not find container \"37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23\": container with ID starting with 37d670a45ff6c82102db15a0447f2522116aed6c3a63230a9d572f9cdc72eb23 not found: ID does not exist"
Jul 22 01:56:21.469552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.469532    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:56:21.475249 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.475228    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fffbfad3-gw-openshift-default-78cf657cc5-q5lkk"]
Jul 22 01:56:21.514096 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514061    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514096 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514094    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514096 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514103    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f54d24bc-343d-4360-894d-c43b9cf63b20-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514339 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514114    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vtk6\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-kube-api-access-2vtk6\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514339 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514124    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514339 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514132    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.514339 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.514140    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f54d24bc-343d-4360-894d-c43b9cf63b20-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:21.990497 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:21.990454    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" path="/var/lib/kubelet/pods/f54d24bc-343d-4360-894d-c43b9cf63b20/volumes"
Jul 22 01:56:45.592981 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.592887    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"]
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593201    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593213    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593223    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593230    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593249    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582329e9-7ef7-4309-ac3c-753195267aa6" containerName="revoke-keys"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593255    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="582329e9-7ef7-4309-ac3c-753195267aa6" containerName="revoke-keys"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593306    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593314    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54d24bc-343d-4360-894d-c43b9cf63b20" containerName="istio-proxy"
Jul 22 01:56:45.593483 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.593321    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="582329e9-7ef7-4309-ac3c-753195267aa6" containerName="revoke-keys"
Jul 22 01:56:45.596436 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.596415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:45.598706 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.598687    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:56:45.603790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.603388    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"]
Jul 22 01:56:45.704693 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.704658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3247df58-vrmqb\" (UID: \"326b30e3-5a6b-4a80-8a30-0cf9f62d9416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:45.805923 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.805888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3247df58-vrmqb\" (UID: \"326b30e3-5a6b-4a80-8a30-0cf9f62d9416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:45.806552 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.806533    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3247df58-vrmqb\" (UID: \"326b30e3-5a6b-4a80-8a30-0cf9f62d9416\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:45.907282 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:45.907251    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:46.037558 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:46.037536    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"]
Jul 22 01:56:46.040019 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:56:46.039979    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod326b30e3_5a6b_4a80_8a30_0cf9f62d9416.slice/crio-2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3 WatchSource:0}: Error finding container 2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3: Status 404 returned error can't find the container with id 2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3
Jul 22 01:56:46.568792 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:46.568705    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb" event={"ID":"326b30e3-5a6b-4a80-8a30-0cf9f62d9416","Type":"ContainerStarted","Data":"8fcfb4cbd56c4cec62f36cab8cf924274c09ba105651887b329c6c8b3c7ece08"}
Jul 22 01:56:46.568792 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:46.568784    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb" event={"ID":"326b30e3-5a6b-4a80-8a30-0cf9f62d9416","Type":"ContainerStarted","Data":"2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3"}
Jul 22 01:56:46.585720 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:46.585670    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb" podStartSLOduration=1.585656886 podStartE2EDuration="1.585656886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:56:46.583667921 +0000 UTC m=+1725.132086044" watchObservedRunningTime="2026-07-22 01:56:46.585656886 +0000 UTC m=+1725.134075011"
Jul 22 01:56:47.575191 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:47.575155    2560 generic.go:356] "Generic (PLEG): container finished" podID="326b30e3-5a6b-4a80-8a30-0cf9f62d9416" containerID="8fcfb4cbd56c4cec62f36cab8cf924274c09ba105651887b329c6c8b3c7ece08" exitCode=0
Jul 22 01:56:47.575632 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:47.575201    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb" event={"ID":"326b30e3-5a6b-4a80-8a30-0cf9f62d9416","Type":"ContainerDied","Data":"8fcfb4cbd56c4cec62f36cab8cf924274c09ba105651887b329c6c8b3c7ece08"}
Jul 22 01:56:48.707809 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:48.707774    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:48.732326 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:48.732297    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca\") pod \"326b30e3-5a6b-4a80-8a30-0cf9f62d9416\" (UID: \"326b30e3-5a6b-4a80-8a30-0cf9f62d9416\") "
Jul 22 01:56:48.732653 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:48.732629    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "326b30e3-5a6b-4a80-8a30-0cf9f62d9416" (UID: "326b30e3-5a6b-4a80-8a30-0cf9f62d9416"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:56:48.834041 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:48.833953    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/326b30e3-5a6b-4a80-8a30-0cf9f62d9416-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:56:49.585088 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:49.585059    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"
Jul 22 01:56:49.585282 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:49.585048    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb" event={"ID":"326b30e3-5a6b-4a80-8a30-0cf9f62d9416","Type":"ContainerDied","Data":"2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3"}
Jul 22 01:56:49.585282 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:56:49.585190    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c1aa170dfca5e900990af57a7f5fa996fb30bfcdd9cad09c91d7d271f620ea3"
Jul 22 01:57:00.675050 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:00.675014    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"]
Jul 22 01:57:00.677816 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:00.677795    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3247df58-vrmqb"]
Jul 22 01:57:01.992884 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:01.992854    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="326b30e3-5a6b-4a80-8a30-0cf9f62d9416" path="/var/lib/kubelet/pods/326b30e3-5a6b-4a80-8a30-0cf9f62d9416/volumes"
Jul 22 01:57:02.843135 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.843105    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:02.843458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.843445    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="326b30e3-5a6b-4a80-8a30-0cf9f62d9416" containerName="revoke-keys"
Jul 22 01:57:02.843509 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.843460    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="326b30e3-5a6b-4a80-8a30-0cf9f62d9416" containerName="revoke-keys"
Jul 22 01:57:02.843545 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.843526    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="326b30e3-5a6b-4a80-8a30-0cf9f62d9416" containerName="revoke-keys"
Jul 22 01:57:02.847798 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.847783    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.850113 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.850083    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9db3f612-openshift-default-dockercfg-w7cgz\""
Jul 22 01:57:02.850241 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.850205    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 01:57:02.857202 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.857178    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:02.961093 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961057    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961127    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7nk8\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961237    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961264 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961257    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961470 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961273    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961470 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:02.961470 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:02.961363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062223 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062186    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062239    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062267    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062293    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7nk8\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062348    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062458    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062515    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062586    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.062734 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062614    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.063147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.063147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062841    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.063147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.063147 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.062999    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.063282 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.063184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.064706 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.064682    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.064811 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.064791    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.070033 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.070014    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.070219 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.070197    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7nk8\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8\") pod \"e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") " pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.158978 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.158945    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:03.293633 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.293562    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:03.295825 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:57:03.295799    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51dda20d_1eec_4b0b_a217_387231d80b11.slice/crio-8e7e9afbab6a8ceeea973e695deef7085fb1e627a916041f925e270897e92dc8 WatchSource:0}: Error finding container 8e7e9afbab6a8ceeea973e695deef7085fb1e627a916041f925e270897e92dc8: Status 404 returned error can't find the container with id 8e7e9afbab6a8ceeea973e695deef7085fb1e627a916041f925e270897e92dc8
Jul 22 01:57:03.298012 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.297973    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:03.298384 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.298040    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:03.298384 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.298071    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:03.650270 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.650239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerStarted","Data":"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"}
Jul 22 01:57:03.650465 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.650291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerStarted","Data":"8e7e9afbab6a8ceeea973e695deef7085fb1e627a916041f925e270897e92dc8"}
Jul 22 01:57:03.668140 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:03.668065    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podStartSLOduration=1.6680439379999998 podStartE2EDuration="1.668043938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:57:03.667667871 +0000 UTC m=+1742.216085996" watchObservedRunningTime="2026-07-22 01:57:03.668043938 +0000 UTC m=+1742.216462066"
Jul 22 01:57:04.159230 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:04.159195    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:04.164468 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:04.164443    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:04.656073 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:04.656044    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:04.656956 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:04.656937    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:16.719898 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.719866    2560 generic.go:356] "Generic (PLEG): container finished" podID="51dda20d-1eec-4b0b-a217-387231d80b11" containerID="bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9" exitCode=0
Jul 22 01:57:16.720372 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.719943    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerDied","Data":"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"}
Jul 22 01:57:16.720372 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.720345    2560 scope.go:117] "RemoveContainer" containerID="bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"
Jul 22 01:57:16.721365 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.721338    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:16.721471 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.721425    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:16.721471 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:16.721457    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:57:17.731106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:17.731059    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerStarted","Data":"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"}
Jul 22 01:57:18.159600 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:18.159561    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:18.160730 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:18.160702    2560 patch_prober.go:28] interesting pod/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 22 01:57:18.160847 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:18.160752    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 22 01:57:19.159785 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:19.159748    2560 patch_prober.go:28] interesting pod/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 22 01:57:19.160172 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:19.159804    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 22 01:57:19.656947 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:19.656909    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:20.160227 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:20.160184    2560 patch_prober.go:28] interesting pod/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 22 01:57:20.160740 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:20.160246    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 22 01:57:21.160080 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:21.160046    2560 patch_prober.go:28] interesting pod/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 22 01:57:21.160271 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:21.160099    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 22 01:57:22.163432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:22.163387    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:22.164318 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:22.164293    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:30.088132 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.088095    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"]
Jul 22 01:57:30.091370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.091353    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:30.093562 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.093536    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:57:30.097790 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.097764    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"]
Jul 22 01:57:30.119610 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.119574    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9db3f612-slqnx\" (UID: \"46204ba0-7992-433c-a161-7cfd70411aec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:30.220528 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.220489    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9db3f612-slqnx\" (UID: \"46204ba0-7992-433c-a161-7cfd70411aec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:30.221104 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.221086    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9db3f612-slqnx\" (UID: \"46204ba0-7992-433c-a161-7cfd70411aec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:30.403250 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.403217    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:30.535230 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.535197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"]
Jul 22 01:57:30.538286 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:57:30.538260    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46204ba0_7992_433c_a161_7cfd70411aec.slice/crio-18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da WatchSource:0}: Error finding container 18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da: Status 404 returned error can't find the container with id 18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da
Jul 22 01:57:30.798561 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:30.798521    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx" event={"ID":"46204ba0-7992-433c-a161-7cfd70411aec","Type":"ContainerStarted","Data":"18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da"}
Jul 22 01:57:31.804518 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:31.804478    2560 generic.go:356] "Generic (PLEG): container finished" podID="46204ba0-7992-433c-a161-7cfd70411aec" containerID="464bbbc06287f6583bd77e280467aca89dfe0f2d315586e3d701c3552633c3c3" exitCode=0
Jul 22 01:57:31.804903 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:31.804566    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx" event={"ID":"46204ba0-7992-433c-a161-7cfd70411aec","Type":"ContainerDied","Data":"464bbbc06287f6583bd77e280467aca89dfe0f2d315586e3d701c3552633c3c3"}
Jul 22 01:57:32.942351 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:32.942323    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:33.046319 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.046287    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca\") pod \"46204ba0-7992-433c-a161-7cfd70411aec\" (UID: \"46204ba0-7992-433c-a161-7cfd70411aec\") "
Jul 22 01:57:33.046696 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.046675    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46204ba0-7992-433c-a161-7cfd70411aec" (UID: "46204ba0-7992-433c-a161-7cfd70411aec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:57:33.147335 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.147307    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46204ba0-7992-433c-a161-7cfd70411aec-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:33.814200 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.814169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx" event={"ID":"46204ba0-7992-433c-a161-7cfd70411aec","Type":"ContainerDied","Data":"18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da"}
Jul 22 01:57:33.814200 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.814193    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"
Jul 22 01:57:33.814475 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:33.814215    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18d5706b58d88a670300193fde6b2feb894b2a11a626f066919b1a26494ec1da"
Jul 22 01:57:45.168966 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:45.168928    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"]
Jul 22 01:57:45.171497 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:45.171469    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9db3f612-slqnx"]
Jul 22 01:57:45.371172 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:45.371141    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:45.371412 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:45.371368    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy" containerID="cri-o://a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96" gracePeriod=30
Jul 22 01:57:45.990481 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:45.990446    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46204ba0-7992-433c-a161-7cfd70411aec" path="/var/lib/kubelet/pods/46204ba0-7992-433c-a161-7cfd70411aec/volumes"
Jul 22 01:57:50.613284 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.613257    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:50.706173 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706074    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706173 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706120    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706173 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706162    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7nk8\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706211    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706241    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706266    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706296    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706324    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706385    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy\") pod \"51dda20d-1eec-4b0b-a217-387231d80b11\" (UID: \"51dda20d-1eec-4b0b-a217-387231d80b11\") "
Jul 22 01:57:50.706479 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706461    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:57:50.706835 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706636    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:57:50.706835 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706704    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:57:50.706835 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706749    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:57:50.706835 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706766    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.706835 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.706784    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.707106 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.707077    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data" (OuterVolumeSpecName: "istio-data") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:57:50.708610 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.708586    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token" (OuterVolumeSpecName: "istio-token") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:57:50.708767 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.708748    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 01:57:50.708885 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.708868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8" (OuterVolumeSpecName: "kube-api-access-p7nk8") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "kube-api-access-p7nk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 01:57:50.708994 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.708981    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51dda20d-1eec-4b0b-a217-387231d80b11" (UID: "51dda20d-1eec-4b0b-a217-387231d80b11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 01:57:50.807566 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807533    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7nk8\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-kube-api-access-p7nk8\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807566 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807560    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51dda20d-1eec-4b0b-a217-387231d80b11-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807566 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807571    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51dda20d-1eec-4b0b-a217-387231d80b11-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807566 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807580    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807827 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807589    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51dda20d-1eec-4b0b-a217-387231d80b11-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807827 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807596    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.807827 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.807604    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51dda20d-1eec-4b0b-a217-387231d80b11-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:57:50.895124 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.895090    2560 generic.go:356] "Generic (PLEG): container finished" podID="51dda20d-1eec-4b0b-a217-387231d80b11" containerID="a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96" exitCode=0
Jul 22 01:57:50.895311 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.895137    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"
Jul 22 01:57:50.895311 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.895184    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerDied","Data":"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"}
Jul 22 01:57:50.895311 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.895241    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl" event={"ID":"51dda20d-1eec-4b0b-a217-387231d80b11","Type":"ContainerDied","Data":"8e7e9afbab6a8ceeea973e695deef7085fb1e627a916041f925e270897e92dc8"}
Jul 22 01:57:50.895311 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.895270    2560 scope.go:117] "RemoveContainer" containerID="a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"
Jul 22 01:57:50.904658 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.904637    2560 scope.go:117] "RemoveContainer" containerID="bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"
Jul 22 01:57:50.912612 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.912587    2560 scope.go:117] "RemoveContainer" containerID="a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"
Jul 22 01:57:50.912867 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:57:50.912845    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96\": container with ID starting with a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96 not found: ID does not exist" containerID="a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"
Jul 22 01:57:50.912929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.912875    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96"} err="failed to get container status \"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96\": rpc error: code = NotFound desc = could not find container \"a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96\": container with ID starting with a5d98d7655775e7fe1dd08fcfe5d24d003b8ab7ebc7afeed5d8aabde4ffc9e96 not found: ID does not exist"
Jul 22 01:57:50.912929 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.912892    2560 scope.go:117] "RemoveContainer" containerID="bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"
Jul 22 01:57:50.913138 ip-10-0-140-155 kubenswrapper[2560]: E0722 01:57:50.913121    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9\": container with ID starting with bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9 not found: ID does not exist" containerID="bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"
Jul 22 01:57:50.913179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.913144    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9"} err="failed to get container status \"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9\": rpc error: code = NotFound desc = could not find container \"bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9\": container with ID starting with bb9c33dc0d84e4b2bff986d533639078de870bd69c0fccd7fbef4949bfec20a9 not found: ID does not exist"
Jul 22 01:57:50.920084 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.920061    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:50.924029 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:50.924010    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9db3f612-openshift-default-55959fccbc-6sqbl"]
Jul 22 01:57:51.991244 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:57:51.991208    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" path="/var/lib/kubelet/pods/51dda20d-1eec-4b0b-a217-387231d80b11/volumes"
Jul 22 01:59:24.469640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.469557    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.469909    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.469921    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.469936    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46204ba0-7992-433c-a161-7cfd70411aec" containerName="revoke-keys"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.469942    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="46204ba0-7992-433c-a161-7cfd70411aec" containerName="revoke-keys"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.470008    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.470016    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="46204ba0-7992-433c-a161-7cfd70411aec" containerName="revoke-keys"
Jul 22 01:59:24.470103 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.470027    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:24.473105 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.473086    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.479346 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.479309    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 01:59:24.479536 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.479306    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9d2d152a-openshift-default-dockercfg-d98ft\""
Jul 22 01:59:24.496179 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.496147    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 01:59:24.502141 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502267 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502267 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502267 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502267 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502322    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502358    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.502432 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.502383    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgr5q\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.602941 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.602906    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.602941 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.602945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603180 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.602962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603180 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.602993    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603180 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603180 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603180 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603461 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603461 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603354    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgr5q\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603571 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603505    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603571 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603645 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603567    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603796 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.603933 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.603882    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.605386 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.605369    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.605476 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.605464    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.610915 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.610874    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.611072 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.611056    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgr5q\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q\") pod \"e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") " pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.789602 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.789511    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:24.923843 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.923815    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 01:59:24.926125 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:59:24.926098    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f573bd4_86cf_492d_ac51_56303de459d1.slice/crio-a5073120f130c42827294e109ab62952f25fb70d1331b5f524b6658ed17aa4e0 WatchSource:0}: Error finding container a5073120f130c42827294e109ab62952f25fb70d1331b5f524b6658ed17aa4e0: Status 404 returned error can't find the container with id a5073120f130c42827294e109ab62952f25fb70d1331b5f524b6658ed17aa4e0
Jul 22 01:59:24.928288 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.928257    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:24.928356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.928323    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:24.928356 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:24.928352    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:25.343860 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:25.343823    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerStarted","Data":"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"}
Jul 22 01:59:25.344029 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:25.343881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerStarted","Data":"a5073120f130c42827294e109ab62952f25fb70d1331b5f524b6658ed17aa4e0"}
Jul 22 01:59:25.365354 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:25.365290    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podStartSLOduration=1.365269564 podStartE2EDuration="1.365269564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:59:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:59:25.364785086 +0000 UTC m=+1883.913203236" watchObservedRunningTime="2026-07-22 01:59:25.365269564 +0000 UTC m=+1883.913687689"
Jul 22 01:59:25.790302 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:25.790270    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:25.795527 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:25.795500    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:26.348494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:26.348466    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:26.349505 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:26.349483    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:39.585321 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.585281    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"]
Jul 22 01:59:39.585833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.585672    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:39.585833 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.585684    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="51dda20d-1eec-4b0b-a217-387231d80b11" containerName="istio-proxy"
Jul 22 01:59:39.588697 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.588673    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:39.591190 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.591170    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 01:59:39.596906 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.596881    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"]
Jul 22 01:59:39.748231 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.748192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg\" (UID: \"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:39.849456 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.849342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg\" (UID: \"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:39.850242 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.850216    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg\" (UID: \"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:39.901411 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:39.901358    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:40.044708 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:40.044645    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"]
Jul 22 01:59:40.047436 ip-10-0-140-155 kubenswrapper[2560]: W0722 01:59:40.047384    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0a849e_d7f0_4c0d_b63e_d146b14bdb1f.slice/crio-f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f WatchSource:0}: Error finding container f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f: Status 404 returned error can't find the container with id f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f
Jul 22 01:59:40.421753 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:40.421714    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg" event={"ID":"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f","Type":"ContainerStarted","Data":"112bf5cfbc0141a27d746c34706bc3a9248084fb0c445eec97fe4077626ecee5"}
Jul 22 01:59:40.421917 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:40.421775    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg" event={"ID":"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f","Type":"ContainerStarted","Data":"f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f"}
Jul 22 01:59:40.439703 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:40.439653    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg" podStartSLOduration=1.4396399340000001 podStartE2EDuration="1.439639934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 01:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 01:59:40.437853963 +0000 UTC m=+1898.986272086" watchObservedRunningTime="2026-07-22 01:59:40.439639934 +0000 UTC m=+1898.988058058"
Jul 22 01:59:41.426102 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:41.425978    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:53670->10.133.0.41:15021: read: connection reset by peer" start-of-body=
Jul 22 01:59:41.426102 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:41.426039    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": read tcp 10.133.0.2:53670->10.133.0.41:15021: read: connection reset by peer"
Jul 22 01:59:41.429259 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:41.429232    2560 generic.go:356] "Generic (PLEG): container finished" podID="0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" containerID="112bf5cfbc0141a27d746c34706bc3a9248084fb0c445eec97fe4077626ecee5" exitCode=0
Jul 22 01:59:41.429345 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:41.429322    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg" event={"ID":"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f","Type":"ContainerDied","Data":"112bf5cfbc0141a27d746c34706bc3a9248084fb0c445eec97fe4077626ecee5"}
Jul 22 01:59:42.435494 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.435457    2560 generic.go:356] "Generic (PLEG): container finished" podID="7f573bd4-86cf-492d-ac51-56303de459d1" containerID="6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34" exitCode=0
Jul 22 01:59:42.435979 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.435533    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerDied","Data":"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"}
Jul 22 01:59:42.436510 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.436098    2560 scope.go:117] "RemoveContainer" containerID="6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"
Jul 22 01:59:42.437223 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.437184    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:42.437319 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.437294    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:42.437370 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.437345    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 01:59:42.593238 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.593211    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:42.675931 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.675896    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca\") pod \"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f\" (UID: \"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f\") "
Jul 22 01:59:42.676285 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.676258    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" (UID: "0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 01:59:42.776830 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:42.776754    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 01:59:43.444274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.444240    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"
Jul 22 01:59:43.444274 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.444267    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg" event={"ID":"0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f","Type":"ContainerDied","Data":"f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f"}
Jul 22 01:59:43.444840 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.444316    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f13424b237eecbb18e406236a9eb5d1497abefd7b665363952be2fba36dcaf3f"
Jul 22 01:59:43.446629 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.446598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerStarted","Data":"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"}
Jul 22 01:59:43.790359 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.790272    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:43.790557 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.790514    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 22 01:59:43.790640 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:43.790594    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 22 01:59:44.790038 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:44.790001    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 22 01:59:44.790484 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:44.790057    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 22 01:59:45.790789 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:45.790753    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 22 01:59:45.791213 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:45.790809    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 22 01:59:46.790883 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:46.790849    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 22 01:59:46.791283 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:46.790904    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 22 01:59:47.793522 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:47.793494    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:47.793986 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:47.793764    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:47.794458 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:47.794441    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 01:59:54.669881 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:54.669841    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"]
Jul 22 01:59:54.671701 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:54.671678    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9d2d152a-lv6hg"]
Jul 22 01:59:55.998548 ip-10-0-140-155 kubenswrapper[2560]: I0722 01:59:55.998513    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" path="/var/lib/kubelet/pods/0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f/volumes"
Jul 22 02:00:00.138810 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.138773    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"]
Jul 22 02:00:00.139295 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.139274    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" containerName="revoke-keys"
Jul 22 02:00:00.139364 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.139300    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" containerName="revoke-keys"
Jul 22 02:00:00.139453 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.139416    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0a849e-d7f0-4c0d-b63e-d146b14bdb1f" containerName="revoke-keys"
Jul 22 02:00:00.141446 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.141425    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"
Jul 22 02:00:00.143806 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.143788    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:00:00.150170 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.150144    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"]
Jul 22 02:00:00.151759 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.151741    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"
Jul 22 02:00:00.485840 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.485817    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"]
Jul 22 02:00:00.488134 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:00:00.488104    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff WatchSource:0}: Error finding container e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff: Status 404 returned error can't find the container with id e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff
Jul 22 02:00:00.527140 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:00.527110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg" event={"ID":"6d00e6d6-351e-482e-bbe7-d060660045d6","Type":"ContainerStarted","Data":"e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff"}
Jul 22 02:00:01.534143 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:01.534107    2560 generic.go:356] "Generic (PLEG): container finished" podID="6d00e6d6-351e-482e-bbe7-d060660045d6" containerID="02e76cbc33964370e3f302deedc63c61734917de61d07c7f73a4ba20ce5e1b21" exitCode=0
Jul 22 02:00:01.534527 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:01.534156    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg" event={"ID":"6d00e6d6-351e-482e-bbe7-d060660045d6","Type":"ContainerDied","Data":"02e76cbc33964370e3f302deedc63c61734917de61d07c7f73a4ba20ce5e1b21"}
Jul 22 02:00:01.561314 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:01.561274    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 02:00:01.561559 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:01.561537    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" containerID="cri-o://2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4" gracePeriod=30
Jul 22 02:00:02.668229 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:02.668206    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"
Jul 22 02:00:02.796724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:02.796641    2560 patch_prober.go:28] interesting pod/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 02:00:02.796724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:02.796700    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 02:00:03.544687 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:03.544658    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg"
Jul 22 02:00:03.544687 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:03.544664    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744760-ssvkg" event={"ID":"6d00e6d6-351e-482e-bbe7-d060660045d6","Type":"ContainerDied","Data":"e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff"}
Jul 22 02:00:03.544943 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:03.544705    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff"
Jul 22 02:00:06.811909 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.811881    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 02:00:06.896169 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896141    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896178    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896210    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896247    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896286    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896571 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896349    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896571 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896375    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896571 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896433    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896571 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896469    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgr5q\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q\") pod \"7f573bd4-86cf-492d-ac51-56303de459d1\" (UID: \"7f573bd4-86cf-492d-ac51-56303de459d1\") "
Jul 22 02:00:06.896571 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896492    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:00:06.896836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896692    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:00:06.896836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896748    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.896836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896767    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.896988 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896851    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:00:06.896988 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.896958    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:00:06.897284 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.897260    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:00:06.898565 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.898532    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:00:06.898672 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.898571    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:00:06.898923 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.898906    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q" (OuterVolumeSpecName: "kube-api-access-hgr5q") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "kube-api-access-hgr5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:00:06.898969 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.898912    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f573bd4-86cf-492d-ac51-56303de459d1" (UID: "7f573bd4-86cf-492d-ac51-56303de459d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:00:06.997956 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997920    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.997956 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997950    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f573bd4-86cf-492d-ac51-56303de459d1-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.997956 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997959    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.997956 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997968    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f573bd4-86cf-492d-ac51-56303de459d1-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.998217 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997977    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgr5q\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-kube-api-access-hgr5q\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.998217 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997986    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f573bd4-86cf-492d-ac51-56303de459d1-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:06.998217 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:06.997995    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f573bd4-86cf-492d-ac51-56303de459d1-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:00:07.565756 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.565720    2560 generic.go:356] "Generic (PLEG): container finished" podID="7f573bd4-86cf-492d-ac51-56303de459d1" containerID="2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4" exitCode=0
Jul 22 02:00:07.565950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.565774    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"
Jul 22 02:00:07.565950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.565800    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerDied","Data":"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"}
Jul 22 02:00:07.565950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.565856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh" event={"ID":"7f573bd4-86cf-492d-ac51-56303de459d1","Type":"ContainerDied","Data":"a5073120f130c42827294e109ab62952f25fb70d1331b5f524b6658ed17aa4e0"}
Jul 22 02:00:07.565950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.565877    2560 scope.go:117] "RemoveContainer" containerID="2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"
Jul 22 02:00:07.574952 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.574934    2560 scope.go:117] "RemoveContainer" containerID="6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"
Jul 22 02:00:07.582784 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.582767    2560 scope.go:117] "RemoveContainer" containerID="2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"
Jul 22 02:00:07.583018 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:07.583000    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4\": container with ID starting with 2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4 not found: ID does not exist" containerID="2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"
Jul 22 02:00:07.583109 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.583029    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4"} err="failed to get container status \"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4\": rpc error: code = NotFound desc = could not find container \"2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4\": container with ID starting with 2c797e34458a4ef5d8b1784ebd1341f32de7df201c756eae9c88a1ce597e8ec4 not found: ID does not exist"
Jul 22 02:00:07.583109 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.583047    2560 scope.go:117] "RemoveContainer" containerID="6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"
Jul 22 02:00:07.583309 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:07.583295    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34\": container with ID starting with 6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34 not found: ID does not exist" containerID="6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"
Jul 22 02:00:07.583356 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.583312    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34"} err="failed to get container status \"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34\": rpc error: code = NotFound desc = could not find container \"6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34\": container with ID starting with 6f22c0fc801932525e47c6123c9996ec3a45f0e28370af95df433b25878dcd34 not found: ID does not exist"
Jul 22 02:00:07.587054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.587035    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 02:00:07.590626 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.590606    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9d2d152a-openshift-default-5c47c48bf9-hdmnh"]
Jul 22 02:00:07.990703 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:07.990672    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" path="/var/lib/kubelet/pods/7f573bd4-86cf-492d-ac51-56303de459d1/volumes"
Jul 22 02:00:15.586497 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:15.586461    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:16.378908 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:16.378876    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:19.942832 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:19.942790    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:19.943316 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:19.943116    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:21.996558 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996527    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996834    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996845    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996864    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996869    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996880    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d00e6d6-351e-482e-bbe7-d060660045d6" containerName="cleanup"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996886    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d00e6d6-351e-482e-bbe7-d060660045d6" containerName="cleanup"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996938    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996945    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d00e6d6-351e-482e-bbe7-d060660045d6" containerName="cleanup"
Jul 22 02:00:21.996928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.996951    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f573bd4-86cf-492d-ac51-56303de459d1" containerName="istio-proxy"
Jul 22 02:00:21.999606 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:21.999590    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.001962 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.001943    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7e4a4670-openshift-default-dockercfg-c5bw2\""
Jul 22 02:00:22.002053 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.001945    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 02:00:22.010412 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.010378    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:00:22.129939 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.129901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130097 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.129951    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130097 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130038    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130097 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130097 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130091    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130265 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130117    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130265 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130145    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130265 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.130265 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.130250    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrng2\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.230724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.230679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.230724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.230727    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrng2\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.230973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.230759    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.230973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.230791    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.230973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.230833    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231152 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231049    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231152 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231116    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231152 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231139    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231152 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231143    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231364 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231150    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231364 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231523 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231578 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.231637 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.231619    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.233159 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.233133    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.233567 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.233549    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.238790 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.238768    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.239255 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.239238    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrng2\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2\") pod \"e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") " pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.310295 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.310202    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:22.450581 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.450537    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:00:22.453006 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:00:22.452974    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200f655f_72e1_43c9_a5dc_88913c33bfa6.slice/crio-d75f3243a5460182d941ecf3937424b43c0aef50129217e035dea2abc1d89a9a WatchSource:0}: Error finding container d75f3243a5460182d941ecf3937424b43c0aef50129217e035dea2abc1d89a9a: Status 404 returned error can't find the container with id d75f3243a5460182d941ecf3937424b43c0aef50129217e035dea2abc1d89a9a
Jul 22 02:00:22.455102 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.455065    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:22.455175 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.455134    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:22.455175 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.455163    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:22.646221 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.646186    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerStarted","Data":"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"}
Jul 22 02:00:22.646419 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.646243    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerStarted","Data":"d75f3243a5460182d941ecf3937424b43c0aef50129217e035dea2abc1d89a9a"}
Jul 22 02:00:22.667474 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:22.667423    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podStartSLOduration=1.667390428 podStartE2EDuration="1.667390428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:00:22.665689171 +0000 UTC m=+1941.214107295" watchObservedRunningTime="2026-07-22 02:00:22.667390428 +0000 UTC m=+1941.215808552"
Jul 22 02:00:23.311321 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:23.311283    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:23.316461 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:23.316437    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:23.650534 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:23.650502    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:23.651350 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:23.651333    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:25.630773 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:25.630743    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:31.404450 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:31.404412    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:35.640314 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:35.640283    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:37.719827 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.719746    2560 generic.go:356] "Generic (PLEG): container finished" podID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerID="914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094" exitCode=0
Jul 22 02:00:37.720224 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.719821    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerDied","Data":"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"}
Jul 22 02:00:37.720224 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.720201    2560 scope.go:117] "RemoveContainer" containerID="914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"
Jul 22 02:00:37.721177 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.721147    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:37.721272 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.721216    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:37.721272 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:37.721248    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:00:38.650984 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:38.650948    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:38.730682 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:38.730647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerStarted","Data":"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"}
Jul 22 02:00:39.311106 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:39.311073    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:39.312263 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:39.312240    2560 patch_prober.go:28] interesting pod/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 22 02:00:39.312370 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:39.312286    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 22 02:00:40.311129 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:40.311092    2560 patch_prober.go:28] interesting pod/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 22 02:00:40.311552 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:40.311152    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 22 02:00:41.311347 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:41.311313    2560 patch_prober.go:28] interesting pod/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 22 02:00:41.311758 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:41.311366    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 22 02:00:42.311386 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:42.311350    2560 patch_prober.go:28] interesting pod/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Jul 22 02:00:42.311798 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:42.311431    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Jul 22 02:00:43.315495 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:43.315454    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:43.315982 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:43.315960    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:43.316566 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:43.316548    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:00:45.649020 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:45.648989    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:46.377707 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:46.377673    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:55.689163 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:00:55.689077    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:00:56.930070 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.930024    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:00:56.932563 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.932544    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.934952 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.934930    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e4a4670\"/\"openshift-service-ca.crt\""
Jul 22 02:00:56.935801 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.935780    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e4a4670\"/\"e2e-lifecycle-model-7e4a4670-kserve-self-signed-certs\""
Jul 22 02:00:56.935801 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.935799    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e4a4670\"/\"default-dockercfg-6pkqx\""
Jul 22 02:00:56.935955 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.935799    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e4a4670\"/\"kube-root-ca.crt\""
Jul 22 02:00:56.942566 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.942541    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:00:56.974686 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974646    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.974837 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974745    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.974837 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.974837 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqb7s\" (UniqueName: \"kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.974837 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974835    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:56.975035 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:56.974884    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076322 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076278    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076389    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076478    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqb7s\" (UniqueName: \"kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076503    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076831 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076798    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076945 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076831    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.076945 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.076887    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.078716 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.078690    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.078868 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.078853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.084991 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.084966    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqb7s\" (UniqueName: \"kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s\") pod \"e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") " pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.242665 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.242578    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:00:57.372606 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.372574    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:00:57.375832 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:00:57.375802    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58b09d5e_6bc5_407d_af68_27e76ad492e1.slice/crio-07b89cbb3925641b4ec8d69d1e7aae3a836874179fff7d36201ce7676aa99ceb WatchSource:0}: Error finding container 07b89cbb3925641b4ec8d69d1e7aae3a836874179fff7d36201ce7676aa99ceb: Status 404 returned error can't find the container with id 07b89cbb3925641b4ec8d69d1e7aae3a836874179fff7d36201ce7676aa99ceb
Jul 22 02:00:57.832246 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.832211    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" event={"ID":"58b09d5e-6bc5-407d-af68-27e76ad492e1","Type":"ContainerStarted","Data":"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"}
Jul 22 02:00:57.832448 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.832257    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" event={"ID":"58b09d5e-6bc5-407d-af68-27e76ad492e1","Type":"ContainerStarted","Data":"07b89cbb3925641b4ec8d69d1e7aae3a836874179fff7d36201ce7676aa99ceb"}
Jul 22 02:00:57.851271 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:00:57.851221    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" podStartSLOduration=1.851205197 podStartE2EDuration="1.851205197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:00:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:00:57.850366132 +0000 UTC m=+1976.398784256" watchObservedRunningTime="2026-07-22 02:00:57.851205197 +0000 UTC m=+1976.399623320"
Jul 22 02:01:01.417458 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:01:01.417422    2560 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d00e6d6_351e_482e_bbe7_d060660045d6.slice/crio-e0ef48aff0626f98556025e5857878afebe7e535a323b825aa9d07f3674284ff\": RecentStats: unable to find data in memory cache]"
Jul 22 02:01:07.242859 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:07.242813    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:01:07.243339 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:07.242879    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:01:07.255491 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:07.255464    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:01:07.889518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:07.889485    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:01:18.408088 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.408049    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"]
Jul 22 02:01:18.410640 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.410619    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:18.412951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.412930    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:01:18.420281 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.420258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"]
Jul 22 02:01:18.573793 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.573759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x\" (UID: \"cd01eee8-ccc3-400f-8fc8-744dc1d48157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:18.675054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.674967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x\" (UID: \"cd01eee8-ccc3-400f-8fc8-744dc1d48157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:18.675616 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.675597    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x\" (UID: \"cd01eee8-ccc3-400f-8fc8-744dc1d48157\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:18.721020 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.720985    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:18.848715 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.848688    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"]
Jul 22 02:01:18.850515 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:01:18.850486    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd01eee8_ccc3_400f_8fc8_744dc1d48157.slice/crio-6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014 WatchSource:0}: Error finding container 6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014: Status 404 returned error can't find the container with id 6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014
Jul 22 02:01:18.933007 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:18.932927    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x" event={"ID":"cd01eee8-ccc3-400f-8fc8-744dc1d48157","Type":"ContainerStarted","Data":"6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014"}
Jul 22 02:01:19.940626 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:19.940587    2560 generic.go:356] "Generic (PLEG): container finished" podID="cd01eee8-ccc3-400f-8fc8-744dc1d48157" containerID="0950f5bc70e19726ff719ac46353a45e21d66bc6d3a50bad8858f3a86f7d5fe6" exitCode=0
Jul 22 02:01:19.941036 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:19.940669    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x" event={"ID":"cd01eee8-ccc3-400f-8fc8-744dc1d48157","Type":"ContainerDied","Data":"0950f5bc70e19726ff719ac46353a45e21d66bc6d3a50bad8858f3a86f7d5fe6"}
Jul 22 02:01:21.074046 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.074019    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:21.097201 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.097172    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca\") pod \"cd01eee8-ccc3-400f-8fc8-744dc1d48157\" (UID: \"cd01eee8-ccc3-400f-8fc8-744dc1d48157\") "
Jul 22 02:01:21.097541 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.097520    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cd01eee8-ccc3-400f-8fc8-744dc1d48157" (UID: "cd01eee8-ccc3-400f-8fc8-744dc1d48157"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:01:21.198456 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.198346    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd01eee8-ccc3-400f-8fc8-744dc1d48157-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:01:21.952213 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.952185    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"
Jul 22 02:01:21.952384 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.952180    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x" event={"ID":"cd01eee8-ccc3-400f-8fc8-744dc1d48157","Type":"ContainerDied","Data":"6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014"}
Jul 22 02:01:21.952384 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:21.952312    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b2b5c951bb262d67068705f5d4e0852e330366af2ea0a0fdd3dc2db1b4d2014"
Jul 22 02:01:38.490385 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:38.490333    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"]
Jul 22 02:01:38.492185 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:38.492158    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7e4a4670-wzf5x"]
Jul 22 02:01:39.990973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:39.990940    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd01eee8-ccc3-400f-8fc8-744dc1d48157" path="/var/lib/kubelet/pods/cd01eee8-ccc3-400f-8fc8-744dc1d48157/volumes"
Jul 22 02:01:45.837813 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:01:45.837778    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:02:00.858272 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:00.858231    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" podUID="58b09d5e-6bc5-407d-af68-27e76ad492e1" containerName="main" containerID="cri-o://ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73" gracePeriod=45
Jul 22 02:02:01.110962 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.110898    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:02:01.151411 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.151351    2560 generic.go:356] "Generic (PLEG): container finished" podID="58b09d5e-6bc5-407d-af68-27e76ad492e1" containerID="ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73" exitCode=0
Jul 22 02:02:01.151411 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.151414    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"
Jul 22 02:02:01.151660 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.151436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" event={"ID":"58b09d5e-6bc5-407d-af68-27e76ad492e1","Type":"ContainerDied","Data":"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"}
Jul 22 02:02:01.151660 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.151489    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn" event={"ID":"58b09d5e-6bc5-407d-af68-27e76ad492e1","Type":"ContainerDied","Data":"07b89cbb3925641b4ec8d69d1e7aae3a836874179fff7d36201ce7676aa99ceb"}
Jul 22 02:02:01.151660 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.151509    2560 scope.go:117] "RemoveContainer" containerID="ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"
Jul 22 02:02:01.160430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.160407    2560 scope.go:117] "RemoveContainer" containerID="ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"
Jul 22 02:02:01.160773 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:02:01.160751    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73\": container with ID starting with ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73 not found: ID does not exist" containerID="ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"
Jul 22 02:02:01.160845 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.160788    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73"} err="failed to get container status \"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73\": rpc error: code = NotFound desc = could not find container \"ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73\": container with ID starting with ccfe5b2d73ff1d7866b43fc36046a7d416df5a724a30ec0d25d072ac19a8ca73 not found: ID does not exist"
Jul 22 02:02:01.164109 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164088    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqb7s\" (UniqueName: \"kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164206 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164129    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164287 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164268    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164345 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164305    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164345 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164335    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164471 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164381    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home" (OuterVolumeSpecName: "home") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:01.164471 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164405    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm\") pod \"58b09d5e-6bc5-407d-af68-27e76ad492e1\" (UID: \"58b09d5e-6bc5-407d-af68-27e76ad492e1\") "
Jul 22 02:02:01.164915 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164641    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:01.164915 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.164655    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:01.165071 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.165033    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.165071 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.165052    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.165071 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.165066    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.166579 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.166541    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s" (OuterVolumeSpecName: "kube-api-access-nqb7s") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "kube-api-access-nqb7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:02:01.166579 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.166529    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm" (OuterVolumeSpecName: "dshm") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:01.166742 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.166583    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58b09d5e-6bc5-407d-af68-27e76ad492e1" (UID: "58b09d5e-6bc5-407d-af68-27e76ad492e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:02:01.265993 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.265954    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58b09d5e-6bc5-407d-af68-27e76ad492e1-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.265993 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.265983    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58b09d5e-6bc5-407d-af68-27e76ad492e1-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.265993 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.265998    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqb7s\" (UniqueName: \"kubernetes.io/projected/58b09d5e-6bc5-407d-af68-27e76ad492e1-kube-api-access-nqb7s\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:01.473218 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.473184    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:02:01.477884 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.477854    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7e4a4670/e2e-lifecycle-model-7e4a4670-kserve-f7f8f9c9f-6qpnn"]
Jul 22 02:02:01.992099 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:01.992069    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b09d5e-6bc5-407d-af68-27e76ad492e1" path="/var/lib/kubelet/pods/58b09d5e-6bc5-407d-af68-27e76ad492e1/volumes"
Jul 22 02:02:02.061910 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:02.061879    2560 scope.go:117] "RemoveContainer" containerID="2f22304de3626a54260333cf736f39af395e80fa696fd778220c5943581931fb"
Jul 22 02:02:02.070171 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:02.070151    2560 scope.go:117] "RemoveContainer" containerID="44c7c459bf7d566b30464e5c878b321291d5143cb4b4d59dc30180fd124f257f"
Jul 22 02:02:29.357154 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:29.357076    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:02:29.357740 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:29.357288    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy" containerID="cri-o://552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314" gracePeriod=30
Jul 22 02:02:34.598840 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.598817    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:02:34.756795 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756728    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.756795 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756765    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.756795 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756781    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756812    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756850    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756916    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrng2\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756945    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.756979    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy\") pod \"200f655f-72e1-43c9-a5dc-88913c33bfa6\" (UID: \"200f655f-72e1-43c9-a5dc-88913c33bfa6\") "
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757106    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757166    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757161    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757254    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/200f655f-72e1-43c9-a5dc-88913c33bfa6-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757269    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.757358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757301    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.757675 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757370    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:34.757783 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.757758    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data" (OuterVolumeSpecName: "istio-data") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:34.759172 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.759145    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:02:34.759293 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.759256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2" (OuterVolumeSpecName: "kube-api-access-rrng2") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "kube-api-access-rrng2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:02:34.759525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.759505    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:02:34.759525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.759514    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token" (OuterVolumeSpecName: "istio-token") pod "200f655f-72e1-43c9-a5dc-88913c33bfa6" (UID: "200f655f-72e1-43c9-a5dc-88913c33bfa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:02:34.858256 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858232    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.858256 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858255    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.858513 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858265    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrng2\" (UniqueName: \"kubernetes.io/projected/200f655f-72e1-43c9-a5dc-88913c33bfa6-kube-api-access-rrng2\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.858513 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858280    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.858513 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858293    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:34.858513 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:34.858302    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/200f655f-72e1-43c9-a5dc-88913c33bfa6-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:02:35.306942 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.306909    2560 generic.go:356] "Generic (PLEG): container finished" podID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerID="552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314" exitCode=0
Jul 22 02:02:35.307107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.306955    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"
Jul 22 02:02:35.307107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.306996    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerDied","Data":"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"}
Jul 22 02:02:35.307107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.307049    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5" event={"ID":"200f655f-72e1-43c9-a5dc-88913c33bfa6","Type":"ContainerDied","Data":"d75f3243a5460182d941ecf3937424b43c0aef50129217e035dea2abc1d89a9a"}
Jul 22 02:02:35.307107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.307070    2560 scope.go:117] "RemoveContainer" containerID="552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"
Jul 22 02:02:35.319067 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.319048    2560 scope.go:117] "RemoveContainer" containerID="914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"
Jul 22 02:02:35.326684 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.326663    2560 scope.go:117] "RemoveContainer" containerID="552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"
Jul 22 02:02:35.326954 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:02:35.326934    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314\": container with ID starting with 552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314 not found: ID does not exist" containerID="552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"
Jul 22 02:02:35.327066 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.326957    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314"} err="failed to get container status \"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314\": rpc error: code = NotFound desc = could not find container \"552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314\": container with ID starting with 552337eed8d8231caff75f4878195b41db00b3cc5dc944024352a8bbbfe64314 not found: ID does not exist"
Jul 22 02:02:35.327066 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.326972    2560 scope.go:117] "RemoveContainer" containerID="914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"
Jul 22 02:02:35.327280 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:02:35.327248    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094\": container with ID starting with 914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094 not found: ID does not exist" containerID="914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"
Jul 22 02:02:35.327376 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.327280    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094"} err="failed to get container status \"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094\": rpc error: code = NotFound desc = could not find container \"914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094\": container with ID starting with 914b7f293042b27ad4266e9908459c35f2c1e2b8325e1dd665ae38a421f79094 not found: ID does not exist"
Jul 22 02:02:35.329658 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.329638    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:02:35.335682 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.335661    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e4a4670-openshift-default-56857fd5d7-jwgt5"]
Jul 22 02:02:35.990771 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:02:35.990739    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" path="/var/lib/kubelet/pods/200f655f-72e1-43c9-a5dc-88913c33bfa6/volumes"
Jul 22 02:03:02.105256 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:02.105223    2560 scope.go:117] "RemoveContainer" containerID="8fcfb4cbd56c4cec62f36cab8cf924274c09ba105651887b329c6c8b3c7ece08"
Jul 22 02:03:26.643181 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643147    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643503    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b09d5e-6bc5-407d-af68-27e76ad492e1" containerName="main"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643514    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b09d5e-6bc5-407d-af68-27e76ad492e1" containerName="main"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643525    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643530    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643544    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643549    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643559    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd01eee8-ccc3-400f-8fc8-744dc1d48157" containerName="revoke-keys"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643564    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd01eee8-ccc3-400f-8fc8-744dc1d48157" containerName="revoke-keys"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643628    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643636    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="200f655f-72e1-43c9-a5dc-88913c33bfa6" containerName="istio-proxy"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643643    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd01eee8-ccc3-400f-8fc8-744dc1d48157" containerName="revoke-keys"
Jul 22 02:03:26.643662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.643649    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b09d5e-6bc5-407d-af68-27e76ad492e1" containerName="main"
Jul 22 02:03:26.645597 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.645580    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.648373 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.648351    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cf99ea-openshift-default-dockercfg-cwfkw\""
Jul 22 02:03:26.648373 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.648365    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 02:03:26.660622 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.660597    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:03:26.770306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770281    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770455 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770455 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770342    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fncps\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770455 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770506    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770560    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770578    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.770749 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.770614    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.871562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.871562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871564    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.871754 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.871885 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.871955 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872013 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.871979    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fncps\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872075 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872003    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872075 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872057    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872184 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872086    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872184 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872234    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872361 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872340    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872450 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.872514 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.872465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.874213 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.874195    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.874311 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.874267    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.881786 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.881763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.881983 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.881964    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fncps\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps\") pod \"e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") " pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:26.955598 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:26.955545    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:27.087864 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.087839    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:03:27.089810 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:03:27.089783    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca758eda_47c4_4013_a85b_c6ee8bb5c9af.slice/crio-6aef3735ace808d5567f4d50997e2c7e941a0296e5aa6d9ada5af424488c1931 WatchSource:0}: Error finding container 6aef3735ace808d5567f4d50997e2c7e941a0296e5aa6d9ada5af424488c1931: Status 404 returned error can't find the container with id 6aef3735ace808d5567f4d50997e2c7e941a0296e5aa6d9ada5af424488c1931
Jul 22 02:03:27.091928 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.091898    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:27.091998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.091960    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:27.091998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.091990    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:27.540696 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.540666    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerStarted","Data":"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"}
Jul 22 02:03:27.540880 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.540712    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerStarted","Data":"6aef3735ace808d5567f4d50997e2c7e941a0296e5aa6d9ada5af424488c1931"}
Jul 22 02:03:27.560245 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.560192    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podStartSLOduration=1.5601728750000001 podStartE2EDuration="1.560172875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:03:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:03:27.557815215 +0000 UTC m=+2126.106233365" watchObservedRunningTime="2026-07-22 02:03:27.560172875 +0000 UTC m=+2126.108591003"
Jul 22 02:03:27.955622 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:27.955590    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:28.960767 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:28.960739    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:29.549962 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:29.549932    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:29.550820 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:29.550802    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:34.832950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.832875    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:03:34.835300 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.835283    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.837667 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.837648    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-207f15-openshift-default-dockercfg-cs5mt\""
Jul 22 02:03:34.846841 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.846819    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:03:34.934877 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.934841    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935028 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.934891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935028 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.934919    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935028 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.934967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935028 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.934989    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935028 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.935013    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935212 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.935118    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935212 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.935162    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:34.935212 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:34.935201    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnkbj\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.035973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.035941    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036158 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.035979    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036158 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036004    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036158 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036158 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036059    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036158 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036081    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnkbj\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036292    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036352    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036699 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036651    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036699 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036680    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.036935 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.036918    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.038254 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.038230    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.038411 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.038382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.048254 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.048232    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnkbj\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.049548 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.049530    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token\") pod \"e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") " pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.145664 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.145629    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:35.277706 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.277679    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:03:35.280261 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:03:35.280233    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39c1e2e0_03d0_40b8_86b9_12823d3f3542.slice/crio-8c6e351368a5d4afb757ea918e11c568b51523f23888c1d7b51f647bb88f528a WatchSource:0}: Error finding container 8c6e351368a5d4afb757ea918e11c568b51523f23888c1d7b51f647bb88f528a: Status 404 returned error can't find the container with id 8c6e351368a5d4afb757ea918e11c568b51523f23888c1d7b51f647bb88f528a
Jul 22 02:03:35.282231 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.282200    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:35.282291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.282265    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:35.282333 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.282320    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:35.579335 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.579247    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerStarted","Data":"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"}
Jul 22 02:03:35.579335 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.579303    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerStarted","Data":"8c6e351368a5d4afb757ea918e11c568b51523f23888c1d7b51f647bb88f528a"}
Jul 22 02:03:35.601308 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:35.601262    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podStartSLOduration=1.601247994 podStartE2EDuration="1.601247994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:03:35.598919446 +0000 UTC m=+2134.147337582" watchObservedRunningTime="2026-07-22 02:03:35.601247994 +0000 UTC m=+2134.149666187"
Jul 22 02:03:36.146136 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:36.146105    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:36.150863 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:36.150837    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:36.584802 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:36.584711    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:36.585682 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:36.585665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:44.551633 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.551596    2560 patch_prober.go:28] interesting pod/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 22 02:03:44.552068 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.551657    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 22 02:03:44.626478 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.626439    2560 generic.go:356] "Generic (PLEG): container finished" podID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerID="5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0" exitCode=0
Jul 22 02:03:44.626684 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.626623    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerDied","Data":"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"}
Jul 22 02:03:44.627250 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.627224    2560 scope.go:117] "RemoveContainer" containerID="5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"
Jul 22 02:03:44.628921 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.628862    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:44.629034 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.628948    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:44.629034 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:44.628995    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:45.634876 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:45.634836    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerStarted","Data":"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"}
Jul 22 02:03:45.955932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:45.955885    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:45.956155 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:45.956069    2560 patch_prober.go:28] interesting pod/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 22 02:03:45.956155 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:45.956117    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 22 02:03:46.956116 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:46.956074    2560 patch_prober.go:28] interesting pod/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 22 02:03:46.956631 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:46.956142    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 22 02:03:47.956740 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:47.956708    2560 patch_prober.go:28] interesting pod/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 22 02:03:47.957120 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:47.956773    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 22 02:03:48.956920 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:48.956838    2560 patch_prober.go:28] interesting pod/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 22 02:03:48.956920 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:48.956894    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 22 02:03:49.660552 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.660520    2560 generic.go:356] "Generic (PLEG): container finished" podID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerID="116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5" exitCode=0
Jul 22 02:03:49.660721 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.660589    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerDied","Data":"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"}
Jul 22 02:03:49.660981 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.660957    2560 scope.go:117] "RemoveContainer" containerID="116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"
Jul 22 02:03:49.661985 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.661958    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:49.662086 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.662044    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:49.662086 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.662083    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:03:49.959967 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.959890    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:49.960338 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.960207    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:49.960961 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:49.960941    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:03:50.667373 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:50.667331    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerStarted","Data":"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"}
Jul 22 02:03:51.146040 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:51.146006    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:51.147156 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:51.147133    2560 patch_prober.go:28] interesting pod/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 02:03:51.147230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:51.147195    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 02:03:51.585749 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:51.585665    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:52.146737 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:52.146705    2560 patch_prober.go:28] interesting pod/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 02:03:52.147136 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:52.146770    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 02:03:53.146471 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:53.146435    2560 patch_prober.go:28] interesting pod/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 02:03:53.146666 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:53.146507    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 02:03:54.146171 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:54.146089    2560 patch_prober.go:28] interesting pod/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 22 02:03:54.146171 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:54.146144    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 22 02:03:55.150680 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:55.150650    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:55.151709 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:55.151689    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:03:56.022655 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.022620    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:03:56.024998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.024979    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.028032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.028010    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cf99ea\"/\"openshift-service-ca.crt\""
Jul 22 02:03:56.028313 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.028010    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cf99ea\"/\"test-model-cf99ea-kserve-self-signed-certs\""
Jul 22 02:03:56.028313 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.028016    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cf99ea\"/\"default-dockercfg-wpr95\""
Jul 22 02:03:56.028313 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.028019    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cf99ea\"/\"kube-root-ca.crt\""
Jul 22 02:03:56.037136 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.037113    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:03:56.149638 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149604    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2htd7\" (UniqueName: \"kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.149820 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149649    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.149820 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.149820 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.149957 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149842    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.149957 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.149860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251181 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251149    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251181 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2htd7\" (UniqueName: \"kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251250    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251290    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251601    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251626    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.251745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.251696    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.253554 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.253523    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.253685 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.253669    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.259470 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.259454    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2htd7\" (UniqueName: \"kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7\") pod \"test-model-cf99ea-kserve-7b769fd7d7-dvtpq\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") " pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.334502 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.334415    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:03:56.468953 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.468930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:03:56.470765 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:03:56.470735    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod535b98a0_b100_4c6c_9837_a338ddc4978b.slice/crio-288684692c025f9de4f1dc2c5076b3c33d15b624e576889350b4d37006cd7dea WatchSource:0}: Error finding container 288684692c025f9de4f1dc2c5076b3c33d15b624e576889350b4d37006cd7dea: Status 404 returned error can't find the container with id 288684692c025f9de4f1dc2c5076b3c33d15b624e576889350b4d37006cd7dea
Jul 22 02:03:56.695688 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.695651    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" event={"ID":"535b98a0-b100-4c6c-9837-a338ddc4978b","Type":"ContainerStarted","Data":"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"}
Jul 22 02:03:56.695891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.695696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" event={"ID":"535b98a0-b100-4c6c-9837-a338ddc4978b","Type":"ContainerStarted","Data":"288684692c025f9de4f1dc2c5076b3c33d15b624e576889350b4d37006cd7dea"}
Jul 22 02:03:56.714070 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:03:56.714023    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" podStartSLOduration=0.714009273 podStartE2EDuration="714.009273ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:03:56.711798399 +0000 UTC m=+2155.260216525" watchObservedRunningTime="2026-07-22 02:03:56.714009273 +0000 UTC m=+2155.262427397"
Jul 22 02:04:02.125586 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:02.125555    2560 scope.go:117] "RemoveContainer" containerID="464bbbc06287f6583bd77e280467aca89dfe0f2d315586e3d701c3552633c3c3"
Jul 22 02:04:06.335244 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:06.335206    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:04:06.335804 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:06.335297    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:04:06.347718 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:06.347696    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:04:06.754480 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:06.754454    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:04:17.220069 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.220037    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:04:17.222647 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.222624    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.226076 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.226054    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-207f15\"/\"kube-root-ca.crt\""
Jul 22 02:04:17.226196 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.226084    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-207f15\"/\"default-dockercfg-pccjf\""
Jul 22 02:04:17.226196 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.226084    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-207f15\"/\"test-model-207f15-kserve-self-signed-certs\""
Jul 22 02:04:17.226196 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.226084    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-207f15\"/\"openshift-service-ca.crt\""
Jul 22 02:04:17.232324 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.232303    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:04:17.338730 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338692    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.338893 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.338893 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.338893 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.338893 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338878    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knv4\" (UniqueName: \"kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.339076 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.338902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440107    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440326 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440132    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440326 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440166    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knv4\" (UniqueName: \"kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440326 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440189    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440326 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440215    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440580 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440635    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.440662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.440657    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.442319 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.442294    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.442574 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.442549    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.447686 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.447663    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knv4\" (UniqueName: \"kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4\") pod \"test-model-207f15-kserve-7f7c66d587-pkdng\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") " pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.532435 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.532341    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:17.659669 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.659645    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:04:17.661730 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:04:17.661704    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad3d3484_9ec4_4201_9048_f026aaa5069f.slice/crio-bbee1a5713a86541711794a6837d7e5c1c9bc6b4b307e6f2e46d1310c2b9d657 WatchSource:0}: Error finding container bbee1a5713a86541711794a6837d7e5c1c9bc6b4b307e6f2e46d1310c2b9d657: Status 404 returned error can't find the container with id bbee1a5713a86541711794a6837d7e5c1c9bc6b4b307e6f2e46d1310c2b9d657
Jul 22 02:04:17.794976 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.794905    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" event={"ID":"ad3d3484-9ec4-4201-9048-f026aaa5069f","Type":"ContainerStarted","Data":"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"}
Jul 22 02:04:17.794976 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.794947    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" event={"ID":"ad3d3484-9ec4-4201-9048-f026aaa5069f","Type":"ContainerStarted","Data":"bbee1a5713a86541711794a6837d7e5c1c9bc6b4b307e6f2e46d1310c2b9d657"}
Jul 22 02:04:17.813363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:17.813323    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" podStartSLOduration=0.813308738 podStartE2EDuration="813.308738ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:04:17.81252607 +0000 UTC m=+2176.360944194" watchObservedRunningTime="2026-07-22 02:04:17.813308738 +0000 UTC m=+2176.361726862"
Jul 22 02:04:27.533530 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:27.533500    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:27.533968 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:27.533556    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:27.546120 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:27.546097    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:04:27.853809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:04:27.853732    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:05:12.851198 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:12.851164    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:05:13.812893 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:13.812852    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:05:27.870764 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:27.870677    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" podUID="535b98a0-b100-4c6c-9837-a338ddc4978b" containerName="main" containerID="cri-o://bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e" gracePeriod=45
Jul 22 02:05:28.115954 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.115930    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:05:28.125556 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.125491    2560 generic.go:356] "Generic (PLEG): container finished" podID="535b98a0-b100-4c6c-9837-a338ddc4978b" containerID="bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e" exitCode=0
Jul 22 02:05:28.125556 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.125523    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"
Jul 22 02:05:28.125676 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.125572    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" event={"ID":"535b98a0-b100-4c6c-9837-a338ddc4978b","Type":"ContainerDied","Data":"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"}
Jul 22 02:05:28.125676 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.125616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq" event={"ID":"535b98a0-b100-4c6c-9837-a338ddc4978b","Type":"ContainerDied","Data":"288684692c025f9de4f1dc2c5076b3c33d15b624e576889350b4d37006cd7dea"}
Jul 22 02:05:28.125676 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.125636    2560 scope.go:117] "RemoveContainer" containerID="bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"
Jul 22 02:05:28.133569 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.133521    2560 scope.go:117] "RemoveContainer" containerID="bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"
Jul 22 02:05:28.134042 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:05:28.134013    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e\": container with ID starting with bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e not found: ID does not exist" containerID="bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"
Jul 22 02:05:28.134132 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.134050    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e"} err="failed to get container status \"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e\": rpc error: code = NotFound desc = could not find container \"bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e\": container with ID starting with bdb16a278e60f06e273d79da903c215bb56a7f988f58242eecf126a658115f8e not found: ID does not exist"
Jul 22 02:05:28.159506 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159480    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159627 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159627 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159613    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2htd7\" (UniqueName: \"kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159706 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159627    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159706 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159648    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159706 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159694    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs\") pod \"535b98a0-b100-4c6c-9837-a338ddc4978b\" (UID: \"535b98a0-b100-4c6c-9837-a338ddc4978b\") "
Jul 22 02:05:28.159852 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159725    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:28.159909 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159862    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache" (OuterVolumeSpecName: "model-cache") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:28.159980 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.159956    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home" (OuterVolumeSpecName: "home") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:28.160042 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.160014    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.160042 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.160030    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.161842 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.161815    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm" (OuterVolumeSpecName: "dshm") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:28.161932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.161842    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7" (OuterVolumeSpecName: "kube-api-access-2htd7") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "kube-api-access-2htd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:05:28.161932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.161844    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "535b98a0-b100-4c6c-9837-a338ddc4978b" (UID: "535b98a0-b100-4c6c-9837-a338ddc4978b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:05:28.261056 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.261025    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.261056 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.261050    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2htd7\" (UniqueName: \"kubernetes.io/projected/535b98a0-b100-4c6c-9837-a338ddc4978b-kube-api-access-2htd7\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.261056 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.261061    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/535b98a0-b100-4c6c-9837-a338ddc4978b-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.261291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.261072    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/535b98a0-b100-4c6c-9837-a338ddc4978b-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:28.446690 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.446657    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:05:28.450548 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.450526    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-cf99ea/test-model-cf99ea-kserve-7b769fd7d7-dvtpq"]
Jul 22 02:05:28.833060 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:28.832960    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" podUID="ad3d3484-9ec4-4201-9048-f026aaa5069f" containerName="main" containerID="cri-o://ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db" gracePeriod=45
Jul 22 02:05:29.083099 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.083037    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:05:29.133206 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.133171    2560 generic.go:356] "Generic (PLEG): container finished" podID="ad3d3484-9ec4-4201-9048-f026aaa5069f" containerID="ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db" exitCode=0
Jul 22 02:05:29.133206 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.133209    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"
Jul 22 02:05:29.133443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.133254    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" event={"ID":"ad3d3484-9ec4-4201-9048-f026aaa5069f","Type":"ContainerDied","Data":"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"}
Jul 22 02:05:29.133443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.133291    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng" event={"ID":"ad3d3484-9ec4-4201-9048-f026aaa5069f","Type":"ContainerDied","Data":"bbee1a5713a86541711794a6837d7e5c1c9bc6b4b307e6f2e46d1310c2b9d657"}
Jul 22 02:05:29.133443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.133308    2560 scope.go:117] "RemoveContainer" containerID="ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"
Jul 22 02:05:29.141763 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.141748    2560 scope.go:117] "RemoveContainer" containerID="ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"
Jul 22 02:05:29.142054 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:05:29.142035    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db\": container with ID starting with ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db not found: ID does not exist" containerID="ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"
Jul 22 02:05:29.142100 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.142063    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db"} err="failed to get container status \"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db\": rpc error: code = NotFound desc = could not find container \"ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db\": container with ID starting with ae916b860c51d2d93a11629d72f18def414a9bd6daae00bd7d45ddfdea9c84db not found: ID does not exist"
Jul 22 02:05:29.168129 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168096    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168129 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168200    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6knv4\" (UniqueName: \"kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168219    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168236    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168264    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm\") pod \"ad3d3484-9ec4-4201-9048-f026aaa5069f\" (UID: \"ad3d3484-9ec4-4201-9048-f026aaa5069f\") "
Jul 22 02:05:29.168589 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168430    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home" (OuterVolumeSpecName: "home") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:29.168589 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168550    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:29.168589 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168558    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:29.168723 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168623    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.168723 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.168643    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.170528 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.170505    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:05:29.170528 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.170515    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm" (OuterVolumeSpecName: "dshm") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:29.170700 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.170551    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4" (OuterVolumeSpecName: "kube-api-access-6knv4") pod "ad3d3484-9ec4-4201-9048-f026aaa5069f" (UID: "ad3d3484-9ec4-4201-9048-f026aaa5069f"). InnerVolumeSpecName "kube-api-access-6knv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:05:29.269803 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.269773    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6knv4\" (UniqueName: \"kubernetes.io/projected/ad3d3484-9ec4-4201-9048-f026aaa5069f-kube-api-access-6knv4\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.269803 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.269801    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad3d3484-9ec4-4201-9048-f026aaa5069f-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.269986 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.269813    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.269986 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.269826    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad3d3484-9ec4-4201-9048-f026aaa5069f-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:29.454192 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.454162    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:05:29.458476 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.458445    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-207f15/test-model-207f15-kserve-7f7c66d587-pkdng"]
Jul 22 02:05:29.997941 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.997906    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535b98a0-b100-4c6c-9837-a338ddc4978b" path="/var/lib/kubelet/pods/535b98a0-b100-4c6c-9837-a338ddc4978b/volumes"
Jul 22 02:05:29.998452 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:29.998432    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad3d3484-9ec4-4201-9048-f026aaa5069f" path="/var/lib/kubelet/pods/ad3d3484-9ec4-4201-9048-f026aaa5069f/volumes"
Jul 22 02:05:35.748616 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:35.748582    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:05:35.749046 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:35.748786    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy" containerID="cri-o://72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c" gracePeriod=30
Jul 22 02:05:40.996029 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:40.996004    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:05:41.073890 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.073831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.073998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.073895    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.073998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.073920    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.073998 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.073982    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074154 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074002    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fncps\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074154 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074022    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074154 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074048    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074299 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074177    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074299 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074212    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert\") pod \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\" (UID: \"ca758eda-47c4-4013-a85b-c6ee8bb5c9af\") "
Jul 22 02:05:41.074299 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074220    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:41.074482 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074462    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:41.074541 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074528    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:41.074603 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074533    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.074852 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.074831    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:05:41.075034 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.075010    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:41.076065 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.076047    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:05:41.076130 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.076114    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:05:41.076429 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.076390    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:05:41.076522 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.076423    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps" (OuterVolumeSpecName: "kube-api-access-fncps") pod "ca758eda-47c4-4013-a85b-c6ee8bb5c9af" (UID: "ca758eda-47c4-4013-a85b-c6ee8bb5c9af"). InnerVolumeSpecName "kube-api-access-fncps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:05:41.175818 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175793    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175818 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175814    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fncps\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-kube-api-access-fncps\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175823    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175831    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175839    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175846    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175854    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.175949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.175862    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca758eda-47c4-4013-a85b-c6ee8bb5c9af-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:05:41.191653 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.191631    2560 generic.go:356] "Generic (PLEG): container finished" podID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerID="72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c" exitCode=0
Jul 22 02:05:41.191746 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.191674    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"
Jul 22 02:05:41.191784 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.191677    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerDied","Data":"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"}
Jul 22 02:05:41.191816 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.191794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf" event={"ID":"ca758eda-47c4-4013-a85b-c6ee8bb5c9af","Type":"ContainerDied","Data":"6aef3735ace808d5567f4d50997e2c7e941a0296e5aa6d9ada5af424488c1931"}
Jul 22 02:05:41.191850 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.191821    2560 scope.go:117] "RemoveContainer" containerID="72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"
Jul 22 02:05:41.200564 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.200549    2560 scope.go:117] "RemoveContainer" containerID="5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"
Jul 22 02:05:41.207479 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.207463    2560 scope.go:117] "RemoveContainer" containerID="72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"
Jul 22 02:05:41.207705 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:05:41.207684    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c\": container with ID starting with 72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c not found: ID does not exist" containerID="72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"
Jul 22 02:05:41.207749 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.207712    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c"} err="failed to get container status \"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c\": rpc error: code = NotFound desc = could not find container \"72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c\": container with ID starting with 72afd92d02318a2768b84670cf80816e2536926817fb7ad206671dcd4933df0c not found: ID does not exist"
Jul 22 02:05:41.207749 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.207727    2560 scope.go:117] "RemoveContainer" containerID="5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"
Jul 22 02:05:41.207948 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:05:41.207928    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0\": container with ID starting with 5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0 not found: ID does not exist" containerID="5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"
Jul 22 02:05:41.208032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.207956    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0"} err="failed to get container status \"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0\": rpc error: code = NotFound desc = could not find container \"5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0\": container with ID starting with 5ec7390763afa01119d0bb5f8e61b35c84b8895b16b3600015a90949d76f60f0 not found: ID does not exist"
Jul 22 02:05:41.214350 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.214316    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:05:41.218889 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.218866    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cf99ea-openshift-default-5675fbb999-4s9vf"]
Jul 22 02:05:41.991778 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:41.991742    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" path="/var/lib/kubelet/pods/ca758eda-47c4-4013-a85b-c6ee8bb5c9af/volumes"
Jul 22 02:05:57.940507 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:57.940474    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:05:57.941077 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:57.940699    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy" containerID="cri-o://602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7" gracePeriod=30
Jul 22 02:05:58.909533 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909503    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:05:58.909847 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909835    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad3d3484-9ec4-4201-9048-f026aaa5069f" containerName="main"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909849    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad3d3484-9ec4-4201-9048-f026aaa5069f" containerName="main"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909858    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909863    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909869    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535b98a0-b100-4c6c-9837-a338ddc4978b" containerName="main"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909876    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="535b98a0-b100-4c6c-9837-a338ddc4978b" containerName="main"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909884    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.909896 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909889    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.910108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909940    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.910108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909950    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="535b98a0-b100-4c6c-9837-a338ddc4978b" containerName="main"
Jul 22 02:05:58.910108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909955    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad3d3484-9ec4-4201-9048-f026aaa5069f" containerName="main"
Jul 22 02:05:58.910108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.909960    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca758eda-47c4-4013-a85b-c6ee8bb5c9af" containerName="istio-proxy"
Jul 22 02:05:58.912985 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.912968    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:58.915485 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.915464    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-353f20-openshift-default-dockercfg-6pt5g\""
Jul 22 02:05:58.924168 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:58.924147    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:05:59.016530 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016505    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016536    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9z2n\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016618    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016668    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.016848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.016731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118014 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.117989    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118019    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118044    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118067    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118132    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118172    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118201    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118475 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9z2n\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118683 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118535    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118683 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118625    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118840 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.118840 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.119041 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.118837    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.120577 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.120553    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.120684 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.120666    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.125796 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.125771    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.126023 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.126004    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9z2n\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n\") pod \"e2e-api-a-353f20-openshift-default-6765f4f649-t44jp\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") " pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.223434 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.223369    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:05:59.352451 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.352287    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:05:59.355158 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:05:59.355132    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcb4491_4e07_4c0a_b87e_82c4c946a9cb.slice/crio-3870feb41ebb58b4e4f9490cbbfee6f1dd2e31fa48a150c6c9ab7425d554c431 WatchSource:0}: Error finding container 3870feb41ebb58b4e4f9490cbbfee6f1dd2e31fa48a150c6c9ab7425d554c431: Status 404 returned error can't find the container with id 3870feb41ebb58b4e4f9490cbbfee6f1dd2e31fa48a150c6c9ab7425d554c431
Jul 22 02:05:59.357147 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.357114    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:05:59.357236 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.357180    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:05:59.357236 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:05:59.357208    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:06:00.283729 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:00.283691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerStarted","Data":"efbd686b8ced6c0d6b9df9a053fb3f8d7e6e50ae5a4ed274c07c6b1aefd38714"}
Jul 22 02:06:00.284101 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:00.283785    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerStarted","Data":"3870feb41ebb58b4e4f9490cbbfee6f1dd2e31fa48a150c6c9ab7425d554c431"}
Jul 22 02:06:00.305136 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:00.305096    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podStartSLOduration=2.305083409 podStartE2EDuration="2.305083409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:06:00.303411148 +0000 UTC m=+2278.851829268" watchObservedRunningTime="2026-07-22 02:06:00.305083409 +0000 UTC m=+2278.853501531"
Jul 22 02:06:01.223568 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:01.223526    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:01.229390 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:01.229358    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:01.288973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:01.288952    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:01.289784 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:01.289766    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:02.150546 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:02.150518    2560 scope.go:117] "RemoveContainer" containerID="112bf5cfbc0141a27d746c34706bc3a9248084fb0c445eec97fe4077626ecee5"
Jul 22 02:06:03.194002 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.193980    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:06:03.252862 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.252797    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.252862 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.252831    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.252862 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.252862    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253095 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.252961    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253095 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.252994    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnkbj\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253095 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253015    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253095 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253034    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253095 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253057    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253339 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253115    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo\") pod \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\" (UID: \"39c1e2e0-03d0-40b8-86b9-12823d3f3542\") "
Jul 22 02:06:03.253339 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253165    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:03.253339 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253231    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:03.253475 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253340    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:03.253524 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253502    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.253524 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253520    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.253640 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253534    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.253640 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253572    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:06:03.253809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.253787    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data" (OuterVolumeSpecName: "istio-data") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:03.255288 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.255264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:06:03.255379 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.255302    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj" (OuterVolumeSpecName: "kube-api-access-fnkbj") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "kube-api-access-fnkbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:06:03.255467 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.255410    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:03.255467 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.255439    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token" (OuterVolumeSpecName: "istio-token") pod "39c1e2e0-03d0-40b8-86b9-12823d3f3542" (UID: "39c1e2e0-03d0-40b8-86b9-12823d3f3542"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:06:03.298114 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.298085    2560 generic.go:356] "Generic (PLEG): container finished" podID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerID="602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7" exitCode=0
Jul 22 02:06:03.298204 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.298128    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"
Jul 22 02:06:03.298204 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.298167    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerDied","Data":"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"}
Jul 22 02:06:03.298275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.298209    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g" event={"ID":"39c1e2e0-03d0-40b8-86b9-12823d3f3542","Type":"ContainerDied","Data":"8c6e351368a5d4afb757ea918e11c568b51523f23888c1d7b51f647bb88f528a"}
Jul 22 02:06:03.298275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.298230    2560 scope.go:117] "RemoveContainer" containerID="602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"
Jul 22 02:06:03.306445 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.306427    2560 scope.go:117] "RemoveContainer" containerID="116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"
Jul 22 02:06:03.313682 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.313667    2560 scope.go:117] "RemoveContainer" containerID="602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"
Jul 22 02:06:03.313899 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:06:03.313884    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7\": container with ID starting with 602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7 not found: ID does not exist" containerID="602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"
Jul 22 02:06:03.313944 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.313906    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7"} err="failed to get container status \"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7\": rpc error: code = NotFound desc = could not find container \"602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7\": container with ID starting with 602ba06a4182cd941af163b541b7dafef6cc5e10e83edc63f880d004c5c6ddf7 not found: ID does not exist"
Jul 22 02:06:03.313944 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.313921    2560 scope.go:117] "RemoveContainer" containerID="116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"
Jul 22 02:06:03.314139 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:06:03.314124    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5\": container with ID starting with 116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5 not found: ID does not exist" containerID="116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"
Jul 22 02:06:03.314181 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.314142    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5"} err="failed to get container status \"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5\": rpc error: code = NotFound desc = could not find container \"116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5\": container with ID starting with 116311762c849111f817d0ffdc8840f39515a4599f110a973166040388c947c5 not found: ID does not exist"
Jul 22 02:06:03.320428 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.320408    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:06:03.332497 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.327950    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-207f15-openshift-default-8678964b9c-bvj7g"]
Jul 22 02:06:03.354007 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.353988    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.354007 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.354007    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.354170 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.354016    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.354170 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.354024    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.354170 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.354032    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnkbj\" (UniqueName: \"kubernetes.io/projected/39c1e2e0-03d0-40b8-86b9-12823d3f3542-kube-api-access-fnkbj\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.354170 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.354040    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39c1e2e0-03d0-40b8-86b9-12823d3f3542-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:03.990975 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:03.990941    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" path="/var/lib/kubelet/pods/39c1e2e0-03d0-40b8-86b9-12823d3f3542/volumes"
Jul 22 02:06:16.290362 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.290328    2560 patch_prober.go:28] interesting pod/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 02:06:16.290797 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.290385    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 02:06:16.369919 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.369890    2560 generic.go:356] "Generic (PLEG): container finished" podID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerID="efbd686b8ced6c0d6b9df9a053fb3f8d7e6e50ae5a4ed274c07c6b1aefd38714" exitCode=0
Jul 22 02:06:16.370046 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.369929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerDied","Data":"efbd686b8ced6c0d6b9df9a053fb3f8d7e6e50ae5a4ed274c07c6b1aefd38714"}
Jul 22 02:06:16.370247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.370232    2560 scope.go:117] "RemoveContainer" containerID="efbd686b8ced6c0d6b9df9a053fb3f8d7e6e50ae5a4ed274c07c6b1aefd38714"
Jul 22 02:06:16.371310 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.371272    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:06:16.371380 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.371359    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:06:16.371441 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:16.371415    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:06:17.377561 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:17.377525    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerStarted","Data":"510c3cb0f0d74d42982640309d0ac635c20755bc4a03320d3259aff4f37a1309"}
Jul 22 02:06:18.223911 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:18.223866    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:18.224515 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:18.224175    2560 patch_prober.go:28] interesting pod/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 02:06:18.224515 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:18.224234    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 02:06:19.224804 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:19.224766    2560 patch_prober.go:28] interesting pod/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 02:06:19.225279 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:19.224832    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 02:06:20.224558 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:20.224513    2560 patch_prober.go:28] interesting pod/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 22 02:06:20.224774 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:20.224585    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 22 02:06:21.228342 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:21.228311    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:21.228812 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:21.228716    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:21.229227 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:21.229210    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:23.555018 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.554987    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"]
Jul 22 02:06:23.555503 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.555479    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:23.555503 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.555496    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:23.555633 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.555600    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:23.555633 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.555613    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:23.557781 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.557763    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:23.560143 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.560123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:06:23.565711 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.565692    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"]
Jul 22 02:06:23.629762 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.629732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-353f20-82pp4\" (UID: \"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:23.730428 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.730381    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-353f20-82pp4\" (UID: \"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:23.730991 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.730974    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-353f20-82pp4\" (UID: \"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:23.868715 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:23.868685    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:24.198800 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:24.198723    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"]
Jul 22 02:06:24.202297 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:06:24.202270    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43f5a23_d72b_46cc_afcc_8f4e3fd6043b.slice/crio-aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8 WatchSource:0}: Error finding container aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8: Status 404 returned error can't find the container with id aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8
Jul 22 02:06:24.409243 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:24.409206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4" event={"ID":"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b","Type":"ContainerStarted","Data":"aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8"}
Jul 22 02:06:25.415150 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:25.415114    2560 generic.go:356] "Generic (PLEG): container finished" podID="b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" containerID="4f9f0624cf662bbbf5db4459ef298b8ace3c7c3b19ad7085ea8abc2b9ee535d6" exitCode=0
Jul 22 02:06:25.415551 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:25.415187    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4" event={"ID":"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b","Type":"ContainerDied","Data":"4f9f0624cf662bbbf5db4459ef298b8ace3c7c3b19ad7085ea8abc2b9ee535d6"}
Jul 22 02:06:26.543246 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:26.543225    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:26.655965 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:26.655927    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca\") pod \"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b\" (UID: \"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b\") "
Jul 22 02:06:26.656274 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:26.656250    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" (UID: "b43f5a23-d72b-46cc-afcc-8f4e3fd6043b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:06:26.756982 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:26.756913    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:27.425121 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:27.425095    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"
Jul 22 02:06:27.425325 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:27.425095    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4" event={"ID":"b43f5a23-d72b-46cc-afcc-8f4e3fd6043b","Type":"ContainerDied","Data":"aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8"}
Jul 22 02:06:27.425325 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:27.425222    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aec6afa328ad542c17a8228f12d2c4c0e66aaa0ec9be866063e903d90189b5b8"
Jul 22 02:06:43.628473 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:43.628443    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"]
Jul 22 02:06:43.630600 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:43.630577    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-353f20-82pp4"]
Jul 22 02:06:43.991620 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:43.991540    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" path="/var/lib/kubelet/pods/b43f5a23-d72b-46cc-afcc-8f4e3fd6043b/volumes"
Jul 22 02:06:50.408975 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.408939    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:06:50.409474 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.409134    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" containerID="cri-o://510c3cb0f0d74d42982640309d0ac635c20755bc4a03320d3259aff4f37a1309" gracePeriod=30
Jul 22 02:06:50.810120 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810041    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"]
Jul 22 02:06:50.810438 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810424    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:50.810504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810439    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39c1e2e0-03d0-40b8-86b9-12823d3f3542" containerName="istio-proxy"
Jul 22 02:06:50.810504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810453    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" containerName="revoke-keys"
Jul 22 02:06:50.810504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810459    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" containerName="revoke-keys"
Jul 22 02:06:50.810612 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.810522    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43f5a23-d72b-46cc-afcc-8f4e3fd6043b" containerName="revoke-keys"
Jul 22 02:06:50.813618 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.813599    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:50.816073 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.816054    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:06:50.821970 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.821946    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"]
Jul 22 02:06:50.959922 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:50.959894    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v\" (UID: \"567aa6c7-807f-48fb-b3a5-e5376b7296de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:51.060763 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.060706    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v\" (UID: \"567aa6c7-807f-48fb-b3a5-e5376b7296de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:51.061286 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.061265    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v\" (UID: \"567aa6c7-807f-48fb-b3a5-e5376b7296de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:51.124926 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.124896    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:51.230676 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.230645    2560 patch_prober.go:28] interesting pod/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 02:06:51.230822 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.230700    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 02:06:51.252422 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.252379    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"]
Jul 22 02:06:51.254663 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:06:51.254637    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567aa6c7_807f_48fb_b3a5_e5376b7296de.slice/crio-ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915 WatchSource:0}: Error finding container ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915: Status 404 returned error can't find the container with id ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915
Jul 22 02:06:51.539267 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.539239    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v" event={"ID":"567aa6c7-807f-48fb-b3a5-e5376b7296de","Type":"ContainerStarted","Data":"f49832919b38e8f2611b76eab668b2938701820ac3957b3fef367b7457ac175f"}
Jul 22 02:06:51.539572 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.539288    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v" event={"ID":"567aa6c7-807f-48fb-b3a5-e5376b7296de","Type":"ContainerStarted","Data":"ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915"}
Jul 22 02:06:51.555511 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:51.555471    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v" podStartSLOduration=1.555456603 podStartE2EDuration="1.555456603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:06:51.554584508 +0000 UTC m=+2330.103002651" watchObservedRunningTime="2026-07-22 02:06:51.555456603 +0000 UTC m=+2330.103874725"
Jul 22 02:06:52.544478 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:52.544440    2560 generic.go:356] "Generic (PLEG): container finished" podID="567aa6c7-807f-48fb-b3a5-e5376b7296de" containerID="f49832919b38e8f2611b76eab668b2938701820ac3957b3fef367b7457ac175f" exitCode=0
Jul 22 02:06:52.544887 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:52.544503    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v" event={"ID":"567aa6c7-807f-48fb-b3a5-e5376b7296de","Type":"ContainerDied","Data":"f49832919b38e8f2611b76eab668b2938701820ac3957b3fef367b7457ac175f"}
Jul 22 02:06:53.677391 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:53.677369    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:53.781823 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:53.781757    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca\") pod \"567aa6c7-807f-48fb-b3a5-e5376b7296de\" (UID: \"567aa6c7-807f-48fb-b3a5-e5376b7296de\") "
Jul 22 02:06:53.782056 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:53.782035    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "567aa6c7-807f-48fb-b3a5-e5376b7296de" (UID: "567aa6c7-807f-48fb-b3a5-e5376b7296de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:06:53.883301 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:53.883272    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/567aa6c7-807f-48fb-b3a5-e5376b7296de-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:54.560668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:54.560640    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"
Jul 22 02:06:54.560860 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:54.560639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v" event={"ID":"567aa6c7-807f-48fb-b3a5-e5376b7296de","Type":"ContainerDied","Data":"ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915"}
Jul 22 02:06:54.560860 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:54.560798    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba2b782e7ced4313e3b076fae8ed808cd3094a8cc572ae6673179173cf9ce915"
Jul 22 02:06:55.567520 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.567481    2560 generic.go:356] "Generic (PLEG): container finished" podID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerID="510c3cb0f0d74d42982640309d0ac635c20755bc4a03320d3259aff4f37a1309" exitCode=0
Jul 22 02:06:55.567860 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.567545    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerDied","Data":"510c3cb0f0d74d42982640309d0ac635c20755bc4a03320d3259aff4f37a1309"}
Jul 22 02:06:55.567860 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.567602    2560 scope.go:117] "RemoveContainer" containerID="efbd686b8ced6c0d6b9df9a053fb3f8d7e6e50ae5a4ed274c07c6b1aefd38714"
Jul 22 02:06:55.646197 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.646179    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:55.796546 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796478    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796546 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796528    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796546 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796570    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796598    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9z2n\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796615    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796946 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796832    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796946 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796847    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:55.796946 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796881    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.796946 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.796941    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs\") pod \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\" (UID: \"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb\") "
Jul 22 02:06:55.797139 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797049    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:55.797139 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797089    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:06:55.797287 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797264    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:55.797348 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797281    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.797348 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797304    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.797348 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797319    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.797555 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.797528    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:55.798813 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.798782    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:06:55.799043 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.799003    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n" (OuterVolumeSpecName: "kube-api-access-r9z2n") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "kube-api-access-r9z2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:06:55.799140 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.799081    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:06:55.799182 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.799157    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" (UID: "6dcb4491-4e07-4c0a-b87e-82c4c946a9cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:06:55.898679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898656    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.898679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898677    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.898807 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898686    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9z2n\" (UniqueName: \"kubernetes.io/projected/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-kube-api-access-r9z2n\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.898807 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898694    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.898807 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898704    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:55.898807 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:55.898711    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:06:56.573966 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:56.573934    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"
Jul 22 02:06:56.573966 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:56.573953    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp" event={"ID":"6dcb4491-4e07-4c0a-b87e-82c4c946a9cb","Type":"ContainerDied","Data":"3870feb41ebb58b4e4f9490cbbfee6f1dd2e31fa48a150c6c9ab7425d554c431"}
Jul 22 02:06:56.574545 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:56.574015    2560 scope.go:117] "RemoveContainer" containerID="510c3cb0f0d74d42982640309d0ac635c20755bc4a03320d3259aff4f37a1309"
Jul 22 02:06:56.592728 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:56.592696    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:06:56.596419 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:56.596381    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-353f20-openshift-default-6765f4f649-t44jp"]
Jul 22 02:06:57.990517 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:06:57.990483    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" path="/var/lib/kubelet/pods/6dcb4491-4e07-4c0a-b87e-82c4c946a9cb/volumes"
Jul 22 02:07:10.884477 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:10.884443    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"]
Jul 22 02:07:10.887236 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:10.887205    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4f2856-rvx5v"]
Jul 22 02:07:11.990324 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:11.990282    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="567aa6c7-807f-48fb-b3a5-e5376b7296de" path="/var/lib/kubelet/pods/567aa6c7-807f-48fb-b3a5-e5376b7296de/volumes"
Jul 22 02:07:18.456838 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.456806    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:07:18.457279 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457264    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:07:18.457352 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457282    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:07:18.457352 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457316    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="567aa6c7-807f-48fb-b3a5-e5376b7296de" containerName="revoke-keys"
Jul 22 02:07:18.457352 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457325    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="567aa6c7-807f-48fb-b3a5-e5376b7296de" containerName="revoke-keys"
Jul 22 02:07:18.457526 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457426    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:07:18.457526 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.457443    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="567aa6c7-807f-48fb-b3a5-e5376b7296de" containerName="revoke-keys"
Jul 22 02:07:18.460478 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.460456    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.462839 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.462818    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 02:07:18.463236 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.463210    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-40b699-openshift-default-dockercfg-mpc25\""
Jul 22 02:07:18.470545 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.470524    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:07:18.546733 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546712    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.546832 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.546832 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.546932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546865    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.546932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546891    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.546932 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546913    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.547039 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546952    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwkfm\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.547039 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.546987    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.547039 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.547007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647547 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647521    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647643 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647551    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647643 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647572    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647643 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647601    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647643 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647627    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647858 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647746    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwkfm\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647858 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647801    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.647858 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647836    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648013 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647887    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648013 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647904    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648013 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.647998    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648171 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.648031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.648270    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.648439 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.648416    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.650017 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.650000    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.650527 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.650510    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.655177 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.655155    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.655305 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.655285    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwkfm\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm\") pod \"e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") " pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.770640 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.770583    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:18.898330 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.898305    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:07:18.900182 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:07:18.900151    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd531303_7561_4604_a234_d5b2f2d53a75.slice/crio-60181c5a9f5058244d5b7a990d9ae1c83bda96a85edca4357a0f96996b821ce5 WatchSource:0}: Error finding container 60181c5a9f5058244d5b7a990d9ae1c83bda96a85edca4357a0f96996b821ce5: Status 404 returned error can't find the container with id 60181c5a9f5058244d5b7a990d9ae1c83bda96a85edca4357a0f96996b821ce5
Jul 22 02:07:18.902171 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.902134    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:18.902247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.902203    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:18.902247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:18.902239    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:19.685629 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:19.685600    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerStarted","Data":"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"}
Jul 22 02:07:19.685990 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:19.685647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerStarted","Data":"60181c5a9f5058244d5b7a990d9ae1c83bda96a85edca4357a0f96996b821ce5"}
Jul 22 02:07:19.704607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:19.704562    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podStartSLOduration=1.7045473100000001 podStartE2EDuration="1.70454731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:07:19.703347166 +0000 UTC m=+2358.251765288" watchObservedRunningTime="2026-07-22 02:07:19.70454731 +0000 UTC m=+2358.252965433"
Jul 22 02:07:19.771305 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:19.771278    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:19.775782 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:19.775764    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:20.690768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:20.690736    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:20.691648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:20.691630    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:36.676826 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.676621    2560 patch_prober.go:28] interesting pod/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:44636->10.133.0.54:15021: read: connection reset by peer" start-of-body=
Jul 22 02:07:36.676826 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.676679    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": read tcp 10.133.0.2:44636->10.133.0.54:15021: read: connection reset by peer"
Jul 22 02:07:36.771573 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.771539    2560 generic.go:356] "Generic (PLEG): container finished" podID="dd531303-7561-4604-a234-d5b2f2d53a75" containerID="3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55" exitCode=0
Jul 22 02:07:36.771668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.771595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerDied","Data":"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"}
Jul 22 02:07:36.771989 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.771974    2560 scope.go:117] "RemoveContainer" containerID="3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"
Jul 22 02:07:36.772958 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.772926    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:36.773052 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.773006    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:36.773112 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:36.773078    2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 22 02:07:37.778119 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:37.778084    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerStarted","Data":"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"}
Jul 22 02:07:38.770857 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:38.770827    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:38.771063 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:38.771042    2560 patch_prober.go:28] interesting pod/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 22 02:07:38.771163 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:38.771097    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 22 02:07:39.771430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:39.771374    2560 patch_prober.go:28] interesting pod/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 22 02:07:39.771809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:39.771447    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 22 02:07:40.771950 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:40.771915    2560 patch_prober.go:28] interesting pod/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 22 02:07:40.772353 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:40.771993    2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 22 02:07:41.774334 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:41.774306    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:41.774824 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:41.774581    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:07:41.775254 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:07:41.775236    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:08:02.196177 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:02.196144    2560 scope.go:117] "RemoveContainer" containerID="0950f5bc70e19726ff719ac46353a45e21d66bc6d3a50bad8858f3a86f7d5fe6"
Jul 22 02:08:05.827704 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.827655    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:08:05.828200 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.828179    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:08:05.828200 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.828199    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:08:05.828359 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.828336    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcb4491-4e07-4c0a-b87e-82c4c946a9cb" containerName="istio-proxy"
Jul 22 02:08:05.832667 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.832643    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.837119 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.836956    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"auth-test-model-40b699-kserve-self-signed-certs\""
Jul 22 02:08:05.837119 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.837096    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"openshift-service-ca.crt\""
Jul 22 02:08:05.837119 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.837062    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"default-dockercfg-m4xnt\""
Jul 22 02:08:05.837379 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.836956    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"kube-root-ca.crt\""
Jul 22 02:08:05.839389 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.839360    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:08:05.898692 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7rr\" (UniqueName: \"kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.898798 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898697    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.898798 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898727    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.898889 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.898889 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898824    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.898889 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.898869    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999583 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999496    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999583 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999541    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7rr\" (UniqueName: \"kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999583 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999864 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999609    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999864 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999768    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:05.999864 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:05.999827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.000110 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.000087    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.000275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.000173    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.000385 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.000293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.002132 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.002104    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.002220 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.002136    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.007826 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.007807    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7rr\" (UniqueName: \"kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr\") pod \"auth-test-model-40b699-kserve-79b886576c-tdtsr\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") " pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.143753 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.143727    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:06.271617 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.271590    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:08:06.273974 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:08:06.273948    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6beba78_6d7e_4efb_a4c0_a9dba2dad10d.slice/crio-ab36dd88f1f191c7e8f5b38681507fef6371c100f455d0bfac8df38d3a7e3f49 WatchSource:0}: Error finding container ab36dd88f1f191c7e8f5b38681507fef6371c100f455d0bfac8df38d3a7e3f49: Status 404 returned error can't find the container with id ab36dd88f1f191c7e8f5b38681507fef6371c100f455d0bfac8df38d3a7e3f49
Jul 22 02:08:06.935146 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.935113    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" event={"ID":"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d","Type":"ContainerStarted","Data":"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"}
Jul 22 02:08:06.935146 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.935152    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" event={"ID":"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d","Type":"ContainerStarted","Data":"ab36dd88f1f191c7e8f5b38681507fef6371c100f455d0bfac8df38d3a7e3f49"}
Jul 22 02:08:06.952494 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:06.952452    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" podStartSLOduration=1.9524388940000001 podStartE2EDuration="1.952438894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:08:06.951308384 +0000 UTC m=+2405.499726508" watchObservedRunningTime="2026-07-22 02:08:06.952438894 +0000 UTC m=+2405.500857016"
Jul 22 02:08:16.144032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:16.143993    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:16.144581 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:16.144050    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:16.156579 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:16.156544    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:16.995515 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:16.995487    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:08:59.806347 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.806313    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:08:59.808635 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.808617    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.812284 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.812255    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"kube-root-ca.crt\""
Jul 22 02:08:59.812444 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.812304    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"openshift-service-ca.crt\""
Jul 22 02:08:59.812444 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.812373    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"default-dockercfg-r4jkw\""
Jul 22 02:08:59.812444 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.812429    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"auth-test-model-1f4af7-kserve-self-signed-certs\""
Jul 22 02:08:59.823381 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.823359    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:08:59.923679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.923792 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923683    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.923792 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923703    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4brq8\" (UniqueName: \"kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.923792 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.923914 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923813    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:08:59.923914 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:08:59.923852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024656 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024630    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024773 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024773 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024699    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4brq8\" (UniqueName: \"kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024773 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024937 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024789    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.024937 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.024835    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.025391 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.025368    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.025580 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.025458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.025650 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.025581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.026924 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.026900    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.027367 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.027348    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.032949 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.032930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4brq8\" (UniqueName: \"kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8\") pod \"auth-test-model-1f4af7-kserve-58c56696ff-h8fpg\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") " pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.118844 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.118823    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:00.245305 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:00.245282    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:09:00.247979 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:09:00.247954    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2de36e4e_0bb8_42e4_9229_5a92e818a005.slice/crio-fcae34f8f5ce4fd75dc24f68028491e73b71e99f285b93f0838daedf6f0da4f7 WatchSource:0}: Error finding container fcae34f8f5ce4fd75dc24f68028491e73b71e99f285b93f0838daedf6f0da4f7: Status 404 returned error can't find the container with id fcae34f8f5ce4fd75dc24f68028491e73b71e99f285b93f0838daedf6f0da4f7
Jul 22 02:09:01.200008 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:01.199967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" event={"ID":"2de36e4e-0bb8-42e4-9229-5a92e818a005","Type":"ContainerStarted","Data":"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"}
Jul 22 02:09:01.200008 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:01.200010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" event={"ID":"2de36e4e-0bb8-42e4-9229-5a92e818a005","Type":"ContainerStarted","Data":"fcae34f8f5ce4fd75dc24f68028491e73b71e99f285b93f0838daedf6f0da4f7"}
Jul 22 02:09:01.219372 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:01.219323    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" podStartSLOduration=2.21930761 podStartE2EDuration="2.21930761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:09:01.217008008 +0000 UTC m=+2459.765426132" watchObservedRunningTime="2026-07-22 02:09:01.21930761 +0000 UTC m=+2459.767725732"
Jul 22 02:09:10.119112 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:10.119076    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:10.119525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:10.119137    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:10.131342 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:10.131316    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:09:10.252484 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:09:10.252460    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:11:13.338603 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:13.338572    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:11:14.377469 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:14.377435    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:11:28.357711 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.357608    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" podUID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" containerName="main" containerID="cri-o://973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909" gracePeriod=45
Jul 22 02:11:28.604929 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.604906    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:11:28.638488 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638431    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638488 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638470    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638634 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638509    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd7rr\" (UniqueName: \"kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638634 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638584    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638634 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638612    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638786 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638672    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs\") pod \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\" (UID: \"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d\") "
Jul 22 02:11:28.638786 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638697    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home" (OuterVolumeSpecName: "home") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:28.638786 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638714    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:28.638942 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638911    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:28.639004 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.638987    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.639041 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.639021    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.640660 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.640639    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr" (OuterVolumeSpecName: "kube-api-access-dd7rr") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "kube-api-access-dd7rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:11:28.641012 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.640998    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:11:28.641086 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.641017    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm" (OuterVolumeSpecName: "dshm") pod "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" (UID: "f6beba78-6d7e-4efb-a4c0-a9dba2dad10d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:28.739830 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.739798    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd7rr\" (UniqueName: \"kubernetes.io/projected/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-kube-api-access-dd7rr\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.739830 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.739827    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.739951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.739836    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.739951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.739844    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:28.888328 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.888302    2560 generic.go:356] "Generic (PLEG): container finished" podID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" containerID="973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909" exitCode=0
Jul 22 02:11:28.888443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.888341    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"
Jul 22 02:11:28.888443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.888389    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" event={"ID":"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d","Type":"ContainerDied","Data":"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"}
Jul 22 02:11:28.888443 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.888436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr" event={"ID":"f6beba78-6d7e-4efb-a4c0-a9dba2dad10d","Type":"ContainerDied","Data":"ab36dd88f1f191c7e8f5b38681507fef6371c100f455d0bfac8df38d3a7e3f49"}
Jul 22 02:11:28.888572 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.888453    2560 scope.go:117] "RemoveContainer" containerID="973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"
Jul 22 02:11:28.898050 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.898035    2560 scope.go:117] "RemoveContainer" containerID="973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"
Jul 22 02:11:28.898476 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:11:28.898456    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909\": container with ID starting with 973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909 not found: ID does not exist" containerID="973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"
Jul 22 02:11:28.898613 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.898591    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909"} err="failed to get container status \"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909\": rpc error: code = NotFound desc = could not find container \"973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909\": container with ID starting with 973ffb909b962031648b8de13b21fc3c96b10d69db43ec72882dea5563203909 not found: ID does not exist"
Jul 22 02:11:28.911590 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.911568    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:11:28.915140 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:28.915120    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/auth-test-model-40b699-kserve-79b886576c-tdtsr"]
Jul 22 02:11:29.396661 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.396627    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" podUID="2de36e4e-0bb8-42e4-9229-5a92e818a005" containerName="main" containerID="cri-o://e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61" gracePeriod=45
Jul 22 02:11:29.624309 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.624287    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:11:29.647411 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647343    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647411 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647374    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647574 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647456    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647574 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647475    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647574 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647525    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4brq8\" (UniqueName: \"kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647574 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647552    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home\") pod \"2de36e4e-0bb8-42e4-9229-5a92e818a005\" (UID: \"2de36e4e-0bb8-42e4-9229-5a92e818a005\") "
Jul 22 02:11:29.647768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647728    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache" (OuterVolumeSpecName: "model-cache") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:29.647940 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647904    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.647940 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647904    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home" (OuterVolumeSpecName: "home") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:29.648077 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.647938    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:29.649816 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.649787    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8" (OuterVolumeSpecName: "kube-api-access-4brq8") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "kube-api-access-4brq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:11:29.649916 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.649829    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm" (OuterVolumeSpecName: "dshm") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:11:29.649916 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.649859    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2de36e4e-0bb8-42e4-9229-5a92e818a005" (UID: "2de36e4e-0bb8-42e4-9229-5a92e818a005"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:11:29.748430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.748392    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.748430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.748429    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2de36e4e-0bb8-42e4-9229-5a92e818a005-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.748562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.748441    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.748562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.748452    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4brq8\" (UniqueName: \"kubernetes.io/projected/2de36e4e-0bb8-42e4-9229-5a92e818a005-kube-api-access-4brq8\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.748562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.748463    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2de36e4e-0bb8-42e4-9229-5a92e818a005-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:11:29.893865 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.893839    2560 generic.go:356] "Generic (PLEG): container finished" podID="2de36e4e-0bb8-42e4-9229-5a92e818a005" containerID="e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61" exitCode=0
Jul 22 02:11:29.893963 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.893879    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"
Jul 22 02:11:29.893963 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.893927    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" event={"ID":"2de36e4e-0bb8-42e4-9229-5a92e818a005","Type":"ContainerDied","Data":"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"}
Jul 22 02:11:29.894070 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.893965    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg" event={"ID":"2de36e4e-0bb8-42e4-9229-5a92e818a005","Type":"ContainerDied","Data":"fcae34f8f5ce4fd75dc24f68028491e73b71e99f285b93f0838daedf6f0da4f7"}
Jul 22 02:11:29.894070 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.893984    2560 scope.go:117] "RemoveContainer" containerID="e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"
Jul 22 02:11:29.902008 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.901992    2560 scope.go:117] "RemoveContainer" containerID="e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"
Jul 22 02:11:29.902236 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:11:29.902220    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61\": container with ID starting with e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61 not found: ID does not exist" containerID="e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"
Jul 22 02:11:29.902274 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.902245    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61"} err="failed to get container status \"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61\": rpc error: code = NotFound desc = could not find container \"e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61\": container with ID starting with e9f0c7dd8f01cf887bb58c75ea58d4f3e62605980d0a05448ab164999a8aac61 not found: ID does not exist"
Jul 22 02:11:29.913728 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.913709    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:11:29.917300 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.917282    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/auth-test-model-1f4af7-kserve-58c56696ff-h8fpg"]
Jul 22 02:11:29.991093 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.991070    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de36e4e-0bb8-42e4-9229-5a92e818a005" path="/var/lib/kubelet/pods/2de36e4e-0bb8-42e4-9229-5a92e818a005/volumes"
Jul 22 02:11:29.991437 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:29.991424    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" path="/var/lib/kubelet/pods/f6beba78-6d7e-4efb-a4c0-a9dba2dad10d/volumes"
Jul 22 02:11:32.926230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926200    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:11:32.926603 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926594    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" containerName="main"
Jul 22 02:11:32.926648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926607    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" containerName="main"
Jul 22 02:11:32.926648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926616    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de36e4e-0bb8-42e4-9229-5a92e818a005" containerName="main"
Jul 22 02:11:32.926648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926622    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de36e4e-0bb8-42e4-9229-5a92e818a005" containerName="main"
Jul 22 02:11:32.926743 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926680    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6beba78-6d7e-4efb-a4c0-a9dba2dad10d" containerName="main"
Jul 22 02:11:32.926743 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.926686    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de36e4e-0bb8-42e4-9229-5a92e818a005" containerName="main"
Jul 22 02:11:32.929825 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.929807    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.933233 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.933208    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"default-dockercfg-m4xnt\""
Jul 22 02:11:32.933353 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.933255    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"kube-root-ca.crt\""
Jul 22 02:11:32.933353 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.933318    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"openshift-service-ca.crt\""
Jul 22 02:11:32.933353 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.933340    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"sub-test-model-40b699-kserve-self-signed-certs\""
Jul 22 02:11:32.940502 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.940482    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:11:32.975732 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975711    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.975839 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.975839 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.975839 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975834    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.975960 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975864    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:32.975960 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:32.975886    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v57x\" (UniqueName: \"kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077215 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077183    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077227    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077247    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v57x\" (UniqueName: \"kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077321    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077363 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077621    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077702 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.077757 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.077711    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.079562 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.079534    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.079781 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.079761    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.085704 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.085681    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v57x\" (UniqueName: \"kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x\") pod \"sub-test-model-40b699-kserve-759866cc79-6crmp\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") " pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.239846 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.239777    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:33.368545 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.368497    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:11:33.370543 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:11:33.370511    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc02d5122_4383_41d0_9ade_3c0d86963f95.slice/crio-cb76a1682cf5a4f6bd2ccfaa323e46562ce1a6909331f57fe5bb6735ea9fbb56 WatchSource:0}: Error finding container cb76a1682cf5a4f6bd2ccfaa323e46562ce1a6909331f57fe5bb6735ea9fbb56: Status 404 returned error can't find the container with id cb76a1682cf5a4f6bd2ccfaa323e46562ce1a6909331f57fe5bb6735ea9fbb56
Jul 22 02:11:33.922711 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.922671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" event={"ID":"c02d5122-4383-41d0-9ade-3c0d86963f95","Type":"ContainerStarted","Data":"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"}
Jul 22 02:11:33.922855 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.922718    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" event={"ID":"c02d5122-4383-41d0-9ade-3c0d86963f95","Type":"ContainerStarted","Data":"cb76a1682cf5a4f6bd2ccfaa323e46562ce1a6909331f57fe5bb6735ea9fbb56"}
Jul 22 02:11:33.940137 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:33.940091    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" podStartSLOduration=1.940076204 podStartE2EDuration="1.940076204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:11:33.938135069 +0000 UTC m=+2612.486553192" watchObservedRunningTime="2026-07-22 02:11:33.940076204 +0000 UTC m=+2612.488494326"
Jul 22 02:11:43.240038 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:43.240004    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:43.240415 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:43.240049    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:43.251491 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:43.251471    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:11:43.977725 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:11:43.977695    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:12:02.729959 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.729930    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:12:02.734501 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.734485    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.737668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.737646    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"kube-root-ca.crt\""
Jul 22 02:12:02.737668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.737656    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"default-dockercfg-r4jkw\""
Jul 22 02:12:02.737851 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.737656    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"sub-test-model-1f4af7-kserve-self-signed-certs\""
Jul 22 02:12:02.737851 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.737662    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"openshift-service-ca.crt\""
Jul 22 02:12:02.743695 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.743673    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:12:02.810426 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810389    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.810575 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810431    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475p4\" (UniqueName: \"kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.810629 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.810667 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810625    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.810705 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.810739 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.810714    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911695    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911871 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911729    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911871 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911748    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911871 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911871 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911824    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.911871 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.911865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-475p4\" (UniqueName: \"kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.912203 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.912184    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.912291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.912257    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.912291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.912268    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.913971 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.913941    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.914219 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.914202    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:02.919380 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:02.919358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-475p4\" (UniqueName: \"kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4\") pod \"sub-test-model-1f4af7-kserve-54d86bcf57-drls8\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") " pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:03.043738 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:03.043669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:03.167631 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:03.167608    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:12:03.169551 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:12:03.169522    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50ea8408_e01d_41ed_8ede_38791db4fd3f.slice/crio-91290092200f8ccc8a57e3de7bfdd3c9cfe8d0b6588285d6f0fbb448b91e2134 WatchSource:0}: Error finding container 91290092200f8ccc8a57e3de7bfdd3c9cfe8d0b6588285d6f0fbb448b91e2134: Status 404 returned error can't find the container with id 91290092200f8ccc8a57e3de7bfdd3c9cfe8d0b6588285d6f0fbb448b91e2134
Jul 22 02:12:04.067008 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:04.066973    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" event={"ID":"50ea8408-e01d-41ed-8ede-38791db4fd3f","Type":"ContainerStarted","Data":"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"}
Jul 22 02:12:04.067479 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:04.067013    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" event={"ID":"50ea8408-e01d-41ed-8ede-38791db4fd3f","Type":"ContainerStarted","Data":"91290092200f8ccc8a57e3de7bfdd3c9cfe8d0b6588285d6f0fbb448b91e2134"}
Jul 22 02:12:04.084611 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:04.084567    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" podStartSLOduration=2.084553901 podStartE2EDuration="2.084553901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:12:04.082613409 +0000 UTC m=+2642.631031533" watchObservedRunningTime="2026-07-22 02:12:04.084553901 +0000 UTC m=+2642.632972023"
Jul 22 02:12:13.044674 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:13.044637    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:13.044674 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:13.044675    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:13.056794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:13.056772    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:13.118493 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:13.118469    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:12:50.563194 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:50.563159    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:12:51.442323 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:12:51.442283    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:13:02.246738 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:02.246702    2560 scope.go:117] "RemoveContainer" containerID="f49832919b38e8f2611b76eab668b2938701820ac3957b3fef367b7457ac175f"
Jul 22 02:13:02.255280 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:02.255256    2560 scope.go:117] "RemoveContainer" containerID="4f9f0624cf662bbbf5db4459ef298b8ace3c7c3b19ad7085ea8abc2b9ee535d6"
Jul 22 02:13:05.583004 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.582958    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" podUID="c02d5122-4383-41d0-9ade-3c0d86963f95" containerName="main" containerID="cri-o://423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316" gracePeriod=45
Jul 22 02:13:05.829296 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.829272    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:13:05.933557 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933534    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933669 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v57x\" (UniqueName: \"kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933669 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933636    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933785 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933687    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933785 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933740    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933861 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933790    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs\") pod \"c02d5122-4383-41d0-9ade-3c0d86963f95\" (UID: \"c02d5122-4383-41d0-9ade-3c0d86963f95\") "
Jul 22 02:13:05.933861 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933807    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache" (OuterVolumeSpecName: "model-cache") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:05.933926 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933871    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:05.933926 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.933912    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home" (OuterVolumeSpecName: "home") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:05.934085 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.934065    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:05.934085 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.934084    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:05.934254 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.934094    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:05.935724 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.935699    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm" (OuterVolumeSpecName: "dshm") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:05.935809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.935769    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:13:05.935809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:05.935772    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x" (OuterVolumeSpecName: "kube-api-access-2v57x") pod "c02d5122-4383-41d0-9ade-3c0d86963f95" (UID: "c02d5122-4383-41d0-9ade-3c0d86963f95"). InnerVolumeSpecName "kube-api-access-2v57x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:13:06.034569 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.034548    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c02d5122-4383-41d0-9ade-3c0d86963f95-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.034569 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.034567    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c02d5122-4383-41d0-9ade-3c0d86963f95-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.034685 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.034577    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v57x\" (UniqueName: \"kubernetes.io/projected/c02d5122-4383-41d0-9ade-3c0d86963f95-kube-api-access-2v57x\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.360662 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.360632    2560 generic.go:356] "Generic (PLEG): container finished" podID="c02d5122-4383-41d0-9ade-3c0d86963f95" containerID="423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316" exitCode=0
Jul 22 02:13:06.360770 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.360682    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"
Jul 22 02:13:06.360770 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.360710    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" event={"ID":"c02d5122-4383-41d0-9ade-3c0d86963f95","Type":"ContainerDied","Data":"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"}
Jul 22 02:13:06.360770 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.360753    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp" event={"ID":"c02d5122-4383-41d0-9ade-3c0d86963f95","Type":"ContainerDied","Data":"cb76a1682cf5a4f6bd2ccfaa323e46562ce1a6909331f57fe5bb6735ea9fbb56"}
Jul 22 02:13:06.360770 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.360771    2560 scope.go:117] "RemoveContainer" containerID="423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"
Jul 22 02:13:06.369472 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.369451    2560 scope.go:117] "RemoveContainer" containerID="423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"
Jul 22 02:13:06.369748 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:13:06.369728    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316\": container with ID starting with 423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316 not found: ID does not exist" containerID="423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"
Jul 22 02:13:06.369819 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.369758    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316"} err="failed to get container status \"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316\": rpc error: code = NotFound desc = could not find container \"423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316\": container with ID starting with 423f263811705b896fc36f3e05f532cef04a0751b8522f717279548f68578316 not found: ID does not exist"
Jul 22 02:13:06.377041 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.377012    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:13:06.381972 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.381950    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/sub-test-model-40b699-kserve-759866cc79-6crmp"]
Jul 22 02:13:06.462459 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.462386    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" podUID="50ea8408-e01d-41ed-8ede-38791db4fd3f" containerName="main" containerID="cri-o://42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4" gracePeriod=45
Jul 22 02:13:06.712866 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.712841    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:13:06.841554 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841526    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841567    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-475p4\" (UniqueName: \"kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841588    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841668 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841615    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841714    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841761    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home\") pod \"50ea8408-e01d-41ed-8ede-38791db4fd3f\" (UID: \"50ea8408-e01d-41ed-8ede-38791db4fd3f\") "
Jul 22 02:13:06.841836 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.841785    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:06.842197 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.842167    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:06.842291 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.842229    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home" (OuterVolumeSpecName: "home") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:06.842434 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.842413    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.842497 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.842441    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.842497 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.842456    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.849358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.844317    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:13:06.849358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.844496    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm" (OuterVolumeSpecName: "dshm") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:13:06.849358 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.845549    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4" (OuterVolumeSpecName: "kube-api-access-475p4") pod "50ea8408-e01d-41ed-8ede-38791db4fd3f" (UID: "50ea8408-e01d-41ed-8ede-38791db4fd3f"). InnerVolumeSpecName "kube-api-access-475p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:13:06.943794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.943738    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50ea8408-e01d-41ed-8ede-38791db4fd3f-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.943794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.943760    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50ea8408-e01d-41ed-8ede-38791db4fd3f-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:06.943794 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:06.943788    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-475p4\" (UniqueName: \"kubernetes.io/projected/50ea8408-e01d-41ed-8ede-38791db4fd3f-kube-api-access-475p4\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:13:07.225306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225254    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:13:07.225632 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225618    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ea8408-e01d-41ed-8ede-38791db4fd3f" containerName="main"
Jul 22 02:13:07.225679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225635    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ea8408-e01d-41ed-8ede-38791db4fd3f" containerName="main"
Jul 22 02:13:07.225679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225663    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c02d5122-4383-41d0-9ade-3c0d86963f95" containerName="main"
Jul 22 02:13:07.225679 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225670    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c02d5122-4383-41d0-9ade-3c0d86963f95" containerName="main"
Jul 22 02:13:07.225773 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225724    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ea8408-e01d-41ed-8ede-38791db4fd3f" containerName="main"
Jul 22 02:13:07.225773 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.225735    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c02d5122-4383-41d0-9ade-3c0d86963f95" containerName="main"
Jul 22 02:13:07.228826 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.228810    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.231270 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.231250    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"kube-root-ca.crt\""
Jul 22 02:13:07.232227 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.232209    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"rate-test-model-40b699-kserve-self-signed-certs\""
Jul 22 02:13:07.232282 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.232228    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"default-dockercfg-m4xnt\""
Jul 22 02:13:07.232282 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.232233    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-40b699\"/\"openshift-service-ca.crt\""
Jul 22 02:13:07.238172 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.238151    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:13:07.348016 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.347996    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.348130 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.348022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.348130 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.348041    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.348130 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.348058    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.348240 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.348150    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.348240 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.348188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktgx\" (UniqueName: \"kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.367597 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.367574    2560 generic.go:356] "Generic (PLEG): container finished" podID="50ea8408-e01d-41ed-8ede-38791db4fd3f" containerID="42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4" exitCode=0
Jul 22 02:13:07.367717 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.367618    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" event={"ID":"50ea8408-e01d-41ed-8ede-38791db4fd3f","Type":"ContainerDied","Data":"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"}
Jul 22 02:13:07.367717 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.367629    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"
Jul 22 02:13:07.367717 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.367640    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8" event={"ID":"50ea8408-e01d-41ed-8ede-38791db4fd3f","Type":"ContainerDied","Data":"91290092200f8ccc8a57e3de7bfdd3c9cfe8d0b6588285d6f0fbb448b91e2134"}
Jul 22 02:13:07.367717 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.367660    2560 scope.go:117] "RemoveContainer" containerID="42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"
Jul 22 02:13:07.375602 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.375577    2560 scope.go:117] "RemoveContainer" containerID="42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"
Jul 22 02:13:07.375871 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:13:07.375843    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4\": container with ID starting with 42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4 not found: ID does not exist" containerID="42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"
Jul 22 02:13:07.375982 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.375878    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4"} err="failed to get container status \"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4\": rpc error: code = NotFound desc = could not find container \"42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4\": container with ID starting with 42681cd791563f127ee93391152145b417c9963a2686e679d8f273fe4c0919e4 not found: ID does not exist"
Jul 22 02:13:07.389465 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.389445    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:13:07.396056 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.396040    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/sub-test-model-1f4af7-kserve-54d86bcf57-drls8"]
Jul 22 02:13:07.448809 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.448914 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448819    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.448914 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.448914 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448876    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktgx\" (UniqueName: \"kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.449068 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.449068 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.448937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.449184 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.449157    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.449262 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.449242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.449321 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.449306    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.450797 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.450781    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.451096 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.451080    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.456608 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.456592    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktgx\" (UniqueName: \"kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx\") pod \"rate-test-model-40b699-kserve-7796cf4d67-qq7q7\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") " pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.538257 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.538206    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:07.663262 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.663238    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:13:07.665102 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:13:07.665077    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ba64323_bb5a_40d5_8a8f_6ab294e366cb.slice/crio-f33b2769ecc753b9b3730d6258acc99b33e02c9efb502098d83e1a8e76bb8842 WatchSource:0}: Error finding container f33b2769ecc753b9b3730d6258acc99b33e02c9efb502098d83e1a8e76bb8842: Status 404 returned error can't find the container with id f33b2769ecc753b9b3730d6258acc99b33e02c9efb502098d83e1a8e76bb8842
Jul 22 02:13:07.990957 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.990917    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50ea8408-e01d-41ed-8ede-38791db4fd3f" path="/var/lib/kubelet/pods/50ea8408-e01d-41ed-8ede-38791db4fd3f/volumes"
Jul 22 02:13:07.991430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:07.991290    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c02d5122-4383-41d0-9ade-3c0d86963f95" path="/var/lib/kubelet/pods/c02d5122-4383-41d0-9ade-3c0d86963f95/volumes"
Jul 22 02:13:08.374198 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:08.374172    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" event={"ID":"3ba64323-bb5a-40d5-8a8f-6ab294e366cb","Type":"ContainerStarted","Data":"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"}
Jul 22 02:13:08.374356 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:08.374210    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" event={"ID":"3ba64323-bb5a-40d5-8a8f-6ab294e366cb","Type":"ContainerStarted","Data":"f33b2769ecc753b9b3730d6258acc99b33e02c9efb502098d83e1a8e76bb8842"}
Jul 22 02:13:08.393708 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:08.393657    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" podStartSLOduration=1.393641952 podStartE2EDuration="1.393641952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:13:08.39117807 +0000 UTC m=+2706.939596192" watchObservedRunningTime="2026-07-22 02:13:08.393641952 +0000 UTC m=+2706.942060074"
Jul 22 02:13:17.538768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:17.538721    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:17.539176 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:17.538781    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:17.550973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:17.550949    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:18.432494 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:18.432465    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:13:29.128004 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.127961    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:13:29.131709 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.131690    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.134934 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.134898    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"rate-test-model-1f4af7-kserve-self-signed-certs\""
Jul 22 02:13:29.134934 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.134913    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"openshift-service-ca.crt\""
Jul 22 02:13:29.135140 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.134944    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"kube-root-ca.crt\""
Jul 22 02:13:29.135140 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.134913    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f4af7\"/\"default-dockercfg-r4jkw\""
Jul 22 02:13:29.141342 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.141317    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:13:29.214526 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214498    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.214620 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214531    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqbw7\" (UniqueName: \"kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.214620 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214579    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.214620 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214615    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.214725 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.214759 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.214731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315664 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315641    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315764 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315688    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315764 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315839 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315821    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315873 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315848    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqbw7\" (UniqueName: \"kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.315907 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.315884    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.316126 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.316098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.316206 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.316147    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.316206 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.316183    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.317872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.317848    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.318222 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.318206    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.324084 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.324068    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqbw7\" (UniqueName: \"kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7\") pod \"rate-test-model-1f4af7-kserve-96b4cc79-qgf5p\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") " pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.441573 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.441511    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:29.570791 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:29.570761    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:13:29.573768 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:13:29.573745    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf322a0f9_f1db_4d78_8fb4_8a093193688f.slice/crio-0d6cc3dac0c0edc21960373dc8bce76a52acf2d5821b6cf9979ade2f94cbc6ff WatchSource:0}: Error finding container 0d6cc3dac0c0edc21960373dc8bce76a52acf2d5821b6cf9979ade2f94cbc6ff: Status 404 returned error can't find the container with id 0d6cc3dac0c0edc21960373dc8bce76a52acf2d5821b6cf9979ade2f94cbc6ff
Jul 22 02:13:30.479786 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:30.479745    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" event={"ID":"f322a0f9-f1db-4d78-8fb4-8a093193688f","Type":"ContainerStarted","Data":"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"}
Jul 22 02:13:30.480176 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:30.479793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" event={"ID":"f322a0f9-f1db-4d78-8fb4-8a093193688f","Type":"ContainerStarted","Data":"0d6cc3dac0c0edc21960373dc8bce76a52acf2d5821b6cf9979ade2f94cbc6ff"}
Jul 22 02:13:30.501134 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:30.501087    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" podStartSLOduration=1.501070809 podStartE2EDuration="1.501070809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:13:30.498978372 +0000 UTC m=+2729.047396496" watchObservedRunningTime="2026-07-22 02:13:30.501070809 +0000 UTC m=+2729.049488931"
Jul 22 02:13:39.442194 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:39.442148    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:39.442194 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:39.442203    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:39.454261 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:39.454232    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:13:39.539890 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:13:39.539860    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:14:35.120840 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:35.120756    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:14:35.997985 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:35.997943    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:14:50.140643 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.140594    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" podUID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" containerName="main" containerID="cri-o://d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5" gracePeriod=45
Jul 22 02:14:50.381130 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.381109    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:14:50.436277 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436223    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436277 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436273    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436309    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436337    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktgx\" (UniqueName: \"kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436368    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436435    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home\") pod \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\" (UID: \"3ba64323-bb5a-40d5-8a8f-6ab294e366cb\") "
Jul 22 02:14:50.436504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436490    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:50.436768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436718    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.436768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436712    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:50.436876 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.436769    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home" (OuterVolumeSpecName: "home") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:50.438329 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.438303    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm" (OuterVolumeSpecName: "dshm") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:50.438430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.438354    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:14:50.438430 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.438387    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx" (OuterVolumeSpecName: "kube-api-access-tktgx") pod "3ba64323-bb5a-40d5-8a8f-6ab294e366cb" (UID: "3ba64323-bb5a-40d5-8a8f-6ab294e366cb"). InnerVolumeSpecName "kube-api-access-tktgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:14:50.537628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.537609    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.537628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.537628    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.537746 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.537637    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktgx\" (UniqueName: \"kubernetes.io/projected/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-kube-api-access-tktgx\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.537746 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.537647    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.537746 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.537655    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba64323-bb5a-40d5-8a8f-6ab294e366cb-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:50.878866 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.878834    2560 generic.go:356] "Generic (PLEG): container finished" podID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" containerID="d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5" exitCode=0
Jul 22 02:14:50.878992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.878883    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"
Jul 22 02:14:50.878992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.878923    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" event={"ID":"3ba64323-bb5a-40d5-8a8f-6ab294e366cb","Type":"ContainerDied","Data":"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"}
Jul 22 02:14:50.878992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.878967    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7" event={"ID":"3ba64323-bb5a-40d5-8a8f-6ab294e366cb","Type":"ContainerDied","Data":"f33b2769ecc753b9b3730d6258acc99b33e02c9efb502098d83e1a8e76bb8842"}
Jul 22 02:14:50.878992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.878994    2560 scope.go:117] "RemoveContainer" containerID="d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"
Jul 22 02:14:50.888122 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.888089    2560 scope.go:117] "RemoveContainer" containerID="d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"
Jul 22 02:14:50.888411 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:14:50.888372    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5\": container with ID starting with d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5 not found: ID does not exist" containerID="d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"
Jul 22 02:14:50.888474 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.888425    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5"} err="failed to get container status \"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5\": rpc error: code = NotFound desc = could not find container \"d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5\": container with ID starting with d4f0f45b08105e8bdbe3ff4cc407de1b4a5fa70c29d468175ba15337263432f5 not found: ID does not exist"
Jul 22 02:14:50.899696 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.899669    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:14:50.903462 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:50.903436    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-40b699/rate-test-model-40b699-kserve-7796cf4d67-qq7q7"]
Jul 22 02:14:51.017787 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.017760    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" podUID="f322a0f9-f1db-4d78-8fb4-8a093193688f" containerName="main" containerID="cri-o://e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9" gracePeriod=45
Jul 22 02:14:51.266587 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.266567    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:14:51.344849 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344827    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.344951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344855    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.344951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344897    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.344951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344920    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.345131 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344953    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqbw7\" (UniqueName: \"kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.345131 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.344989    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs\") pod \"f322a0f9-f1db-4d78-8fb4-8a093193688f\" (UID: \"f322a0f9-f1db-4d78-8fb4-8a093193688f\") "
Jul 22 02:14:51.345131 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345099    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache" (OuterVolumeSpecName: "model-cache") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:51.345278 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345134    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:51.345278 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345206    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home" (OuterVolumeSpecName: "home") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:51.345278 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345256    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-home\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.345278 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345267    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-model-cache\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.345278 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.345276    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-tmp-dir\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.346891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.346868    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm" (OuterVolumeSpecName: "dshm") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:14:51.346989 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.346912    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 02:14:51.346989 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.346924    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7" (OuterVolumeSpecName: "kube-api-access-cqbw7") pod "f322a0f9-f1db-4d78-8fb4-8a093193688f" (UID: "f322a0f9-f1db-4d78-8fb4-8a093193688f"). InnerVolumeSpecName "kube-api-access-cqbw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:14:51.446628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.446574    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f322a0f9-f1db-4d78-8fb4-8a093193688f-dshm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.446628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.446596    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqbw7\" (UniqueName: \"kubernetes.io/projected/f322a0f9-f1db-4d78-8fb4-8a093193688f-kube-api-access-cqbw7\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.446628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.446608    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f322a0f9-f1db-4d78-8fb4-8a093193688f-tls-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:51.887403 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.887371    2560 generic.go:356] "Generic (PLEG): container finished" podID="f322a0f9-f1db-4d78-8fb4-8a093193688f" containerID="e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9" exitCode=0
Jul 22 02:14:51.887525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.887439    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"
Jul 22 02:14:51.887525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.887463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" event={"ID":"f322a0f9-f1db-4d78-8fb4-8a093193688f","Type":"ContainerDied","Data":"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"}
Jul 22 02:14:51.887525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.887510    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p" event={"ID":"f322a0f9-f1db-4d78-8fb4-8a093193688f","Type":"ContainerDied","Data":"0d6cc3dac0c0edc21960373dc8bce76a52acf2d5821b6cf9979ade2f94cbc6ff"}
Jul 22 02:14:51.887661 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.887530    2560 scope.go:117] "RemoveContainer" containerID="e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"
Jul 22 02:14:51.896731 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.896700    2560 scope.go:117] "RemoveContainer" containerID="e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"
Jul 22 02:14:51.897002 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:14:51.896983    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9\": container with ID starting with e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9 not found: ID does not exist" containerID="e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"
Jul 22 02:14:51.897047 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.897007    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9"} err="failed to get container status \"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9\": rpc error: code = NotFound desc = could not find container \"e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9\": container with ID starting with e03c910ff045213357d919eec2c213c0486d1cf06e37d7aaf79a90de21160cc9 not found: ID does not exist"
Jul 22 02:14:51.907891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.907867    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:14:51.911666 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.911641    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f4af7/rate-test-model-1f4af7-kserve-96b4cc79-qgf5p"]
Jul 22 02:14:51.990973 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.990948    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" path="/var/lib/kubelet/pods/3ba64323-bb5a-40d5-8a8f-6ab294e366cb/volumes"
Jul 22 02:14:51.991314 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:51.991301    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f322a0f9-f1db-4d78-8fb4-8a093193688f" path="/var/lib/kubelet/pods/f322a0f9-f1db-4d78-8fb4-8a093193688f/volumes"
Jul 22 02:14:52.650074 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650040    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7655-140270238246928"]
Jul 22 02:14:52.650414 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650373    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" containerName="main"
Jul 22 02:14:52.650414 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650385    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" containerName="main"
Jul 22 02:14:52.650504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650428    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f322a0f9-f1db-4d78-8fb4-8a093193688f" containerName="main"
Jul 22 02:14:52.650504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650435    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f322a0f9-f1db-4d78-8fb4-8a093193688f" containerName="main"
Jul 22 02:14:52.650504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650492    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f322a0f9-f1db-4d78-8fb4-8a093193688f" containerName="main"
Jul 22 02:14:52.650504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.650499    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba64323-bb5a-40d5-8a8f-6ab294e366cb" containerName="main"
Jul 22 02:14:52.654690 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.654671    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:52.657261 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.657232    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h7vgx\""
Jul 22 02:14:52.657420 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.657276    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-140270238246928"]
Jul 22 02:14:52.755785 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.755761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftxx\" (UniqueName: \"kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx\") pod \"test-curl-7655-140270238246928\" (UID: \"9070ecad-03b3-410e-9d8b-8697f7ab7148\") " pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:52.856747 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.856718    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftxx\" (UniqueName: \"kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx\") pod \"test-curl-7655-140270238246928\" (UID: \"9070ecad-03b3-410e-9d8b-8697f7ab7148\") " pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:52.865066 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.865042    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftxx\" (UniqueName: \"kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx\") pod \"test-curl-7655-140270238246928\" (UID: \"9070ecad-03b3-410e-9d8b-8697f7ab7148\") " pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:52.965721 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:52.965661    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:53.093550 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:53.093506    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7655-140270238246928"]
Jul 22 02:14:53.098767 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:53.098742    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 02:14:53.899887 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:53.899847    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-140270238246928" event={"ID":"9070ecad-03b3-410e-9d8b-8697f7ab7148","Type":"ContainerStarted","Data":"63dbbfd33dbc351fb725008672bb5ffcf00cf171c8d9345645ae22faacc2e640"}
Jul 22 02:14:54.906951 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:54.906913    2560 generic.go:356] "Generic (PLEG): container finished" podID="9070ecad-03b3-410e-9d8b-8697f7ab7148" containerID="6945fae7e909088f4a648dee5e4a2a8423627443d327cd4ebaa07797b55f4e17" exitCode=0
Jul 22 02:14:54.907297 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:54.907000    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7655-140270238246928" event={"ID":"9070ecad-03b3-410e-9d8b-8697f7ab7148","Type":"ContainerDied","Data":"6945fae7e909088f4a648dee5e4a2a8423627443d327cd4ebaa07797b55f4e17"}
Jul 22 02:14:56.042846 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.042822    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:56.081959 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.081935    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7655-140270238246928_9070ecad-03b3-410e-9d8b-8697f7ab7148/test-curl-7655-140270238246928/0.log"
Jul 22 02:14:56.085097 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.085072    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gftxx\" (UniqueName: \"kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx\") pod \"9070ecad-03b3-410e-9d8b-8697f7ab7148\" (UID: \"9070ecad-03b3-410e-9d8b-8697f7ab7148\") "
Jul 22 02:14:56.087088 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.087058    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx" (OuterVolumeSpecName: "kube-api-access-gftxx") pod "9070ecad-03b3-410e-9d8b-8697f7ab7148" (UID: "9070ecad-03b3-410e-9d8b-8697f7ab7148"). InnerVolumeSpecName "kube-api-access-gftxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:14:56.106150 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.106127    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7655-140270238246928"]
Jul 22 02:14:56.110122 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.110101    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7655-140270238246928"]
Jul 22 02:14:56.186124 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.186072    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gftxx\" (UniqueName: \"kubernetes.io/projected/9070ecad-03b3-410e-9d8b-8697f7ab7148-kube-api-access-gftxx\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:56.302108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.302075    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"]
Jul 22 02:14:56.302518 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.302497    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" containerName="test-curl-7655-140270238246928"
Jul 22 02:14:56.302624 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.302519    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" containerName="test-curl-7655-140270238246928"
Jul 22 02:14:56.302680 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.302663    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" containerName="test-curl-7655-140270238246928"
Jul 22 02:14:56.305789 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.305768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:56.308312 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.308290    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:14:56.312351 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.312327    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"]
Jul 22 02:14:56.388528 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.388496    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm\" (UID: \"f0957d9c-3591-4b02-86b2-04dce1b1f210\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:56.489112 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.489049    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm\" (UID: \"f0957d9c-3591-4b02-86b2-04dce1b1f210\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:56.489648 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.489631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm\" (UID: \"f0957d9c-3591-4b02-86b2-04dce1b1f210\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:56.617087 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.617063    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:56.739657 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.739594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"]
Jul 22 02:14:56.742419 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:14:56.742379    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0957d9c_3591_4b02_86b2_04dce1b1f210.slice/crio-72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64 WatchSource:0}: Error finding container 72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64: Status 404 returned error can't find the container with id 72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64
Jul 22 02:14:56.918144 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.918120    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm" event={"ID":"f0957d9c-3591-4b02-86b2-04dce1b1f210","Type":"ContainerStarted","Data":"72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64"}
Jul 22 02:14:56.919351 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.919335    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63dbbfd33dbc351fb725008672bb5ffcf00cf171c8d9345645ae22faacc2e640"
Jul 22 02:14:56.919454 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:56.919367    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:56.928662 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:14:56.928635    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-140270238246928\" is forbidden: User \"system:node:ip-10-0-140-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-155.ec2.internal' and this object" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:57.926157 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:57.926123    2560 generic.go:356] "Generic (PLEG): container finished" podID="f0957d9c-3591-4b02-86b2-04dce1b1f210" containerID="9a3c8aede5ccfbd3002122c164a9ec92bc92e64a702fe6b2a1d3350dd8512fc4" exitCode=0
Jul 22 02:14:57.926510 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:57.926205    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm" event={"ID":"f0957d9c-3591-4b02-86b2-04dce1b1f210","Type":"ContainerDied","Data":"9a3c8aede5ccfbd3002122c164a9ec92bc92e64a702fe6b2a1d3350dd8512fc4"}
Jul 22 02:14:57.928174 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:14:57.928146    2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7655-140270238246928\" is forbidden: User \"system:node:ip-10-0-140-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-155.ec2.internal' and this object" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" pod="opendatahub/test-curl-7655-140270238246928"
Jul 22 02:14:57.990032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:57.990007    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9070ecad-03b3-410e-9d8b-8697f7ab7148" path="/var/lib/kubelet/pods/9070ecad-03b3-410e-9d8b-8697f7ab7148/volumes"
Jul 22 02:14:59.063529 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.063508    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:59.106197 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.106172    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca\") pod \"f0957d9c-3591-4b02-86b2-04dce1b1f210\" (UID: \"f0957d9c-3591-4b02-86b2-04dce1b1f210\") "
Jul 22 02:14:59.106520 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.106503    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0957d9c-3591-4b02-86b2-04dce1b1f210" (UID: "f0957d9c-3591-4b02-86b2-04dce1b1f210"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:14:59.206828 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.206776    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0957d9c-3591-4b02-86b2-04dce1b1f210-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:14:59.937500 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.937468    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"
Jul 22 02:14:59.937636 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.937500    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm" event={"ID":"f0957d9c-3591-4b02-86b2-04dce1b1f210","Type":"ContainerDied","Data":"72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64"}
Jul 22 02:14:59.937636 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:14:59.937589    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72fa517534e042ca88b69c2247df142420dd09ddc8595fe8d3cd3b5d791ede64"
Jul 22 02:15:00.138068 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.138045    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"]
Jul 22 02:15:00.138420 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.138361    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0957d9c-3591-4b02-86b2-04dce1b1f210" containerName="revoke-keys"
Jul 22 02:15:00.138420 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.138373    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0957d9c-3591-4b02-86b2-04dce1b1f210" containerName="revoke-keys"
Jul 22 02:15:00.138510 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.138455    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0957d9c-3591-4b02-86b2-04dce1b1f210" containerName="revoke-keys"
Jul 22 02:15:00.141390 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.141373    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"
Jul 22 02:15:00.143990 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.143968    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:15:00.144628 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.144606    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"]
Jul 22 02:15:00.148446 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.148421    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"]
Jul 22 02:15:00.148958 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.148930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"
Jul 22 02:15:00.152544 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.152523    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"]
Jul 22 02:15:00.152848 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.152829    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"
Jul 22 02:15:00.154854 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.154355    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"]
Jul 22 02:15:00.156942 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.156920    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"
Jul 22 02:15:00.158364 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.157972    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"]
Jul 22 02:15:00.166087 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.166070    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"
Jul 22 02:15:00.196283 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.196255    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"
Jul 22 02:15:00.319065 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.319036    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"]
Jul 22 02:15:00.323522 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:15:00.323471    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27222cf_4271_4347_9efc_e1c58d08a1a7.slice/crio-ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef WatchSource:0}: Error finding container ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef: Status 404 returned error can't find the container with id ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef
Jul 22 02:15:00.362305 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.362284    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"]
Jul 22 02:15:00.363593 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:15:00.363572    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d1b8f4_8cc3_46e8_a0d5_becb07d07395.slice/crio-da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b WatchSource:0}: Error finding container da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b: Status 404 returned error can't find the container with id da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b
Jul 22 02:15:00.535372 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.535349    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"]
Jul 22 02:15:00.618938 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:15:00.618914    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4159c8_61f5_4665_8a28_e55f0da713c8.slice/crio-eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57 WatchSource:0}: Error finding container eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57: Status 404 returned error can't find the container with id eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57
Jul 22 02:15:00.944797 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.944763    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg" event={"ID":"e5d1b8f4-8cc3-46e8-a0d5-becb07d07395","Type":"ContainerStarted","Data":"95119c0c60a297d85fd5f39f073e63c72955e3377783f56086accdd7b131cfe1"}
Jul 22 02:15:00.944908 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.944825    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg" event={"ID":"e5d1b8f4-8cc3-46e8-a0d5-becb07d07395","Type":"ContainerStarted","Data":"da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b"}
Jul 22 02:15:00.946210 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.946188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2" event={"ID":"ed4159c8-61f5-4665-8a28-e55f0da713c8","Type":"ContainerStarted","Data":"612cfe588c527862cc2fbe20f86e2aa526d1b6ef6a7babd3ec63bbc3351364cf"}
Jul 22 02:15:00.946306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.946237    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2" event={"ID":"ed4159c8-61f5-4665-8a28-e55f0da713c8","Type":"ContainerStarted","Data":"eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57"}
Jul 22 02:15:00.947684 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.947660    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx" event={"ID":"c27222cf-4271-4347-9efc-e1c58d08a1a7","Type":"ContainerStarted","Data":"03af4ce0433f093d15bf781260f387df840de50516ceebb9be1108493f8ee99a"}
Jul 22 02:15:00.947768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.947706    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx" event={"ID":"c27222cf-4271-4347-9efc-e1c58d08a1a7","Type":"ContainerStarted","Data":"ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef"}
Jul 22 02:15:00.960841 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.960800    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg" podStartSLOduration=0.960787312 podStartE2EDuration="960.787312ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:00.959938755 +0000 UTC m=+2819.508356899" watchObservedRunningTime="2026-07-22 02:15:00.960787312 +0000 UTC m=+2819.509205435"
Jul 22 02:15:00.973948 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.973909    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2" podStartSLOduration=0.973895094 podStartE2EDuration="973.895094ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:00.973536198 +0000 UTC m=+2819.521954321" watchObservedRunningTime="2026-07-22 02:15:00.973895094 +0000 UTC m=+2819.522313196"
Jul 22 02:15:00.988302 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:00.988268    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx" podStartSLOduration=0.988257139 podStartE2EDuration="988.257139ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:15:00.987390253 +0000 UTC m=+2819.535808374" watchObservedRunningTime="2026-07-22 02:15:00.988257139 +0000 UTC m=+2819.536675260"
Jul 22 02:15:01.955099 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.955061    2560 generic.go:356] "Generic (PLEG): container finished" podID="e5d1b8f4-8cc3-46e8-a0d5-becb07d07395" containerID="95119c0c60a297d85fd5f39f073e63c72955e3377783f56086accdd7b131cfe1" exitCode=0
Jul 22 02:15:01.955479 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.955123    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg" event={"ID":"e5d1b8f4-8cc3-46e8-a0d5-becb07d07395","Type":"ContainerDied","Data":"95119c0c60a297d85fd5f39f073e63c72955e3377783f56086accdd7b131cfe1"}
Jul 22 02:15:01.956745 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.956721    2560 generic.go:356] "Generic (PLEG): container finished" podID="ed4159c8-61f5-4665-8a28-e55f0da713c8" containerID="612cfe588c527862cc2fbe20f86e2aa526d1b6ef6a7babd3ec63bbc3351364cf" exitCode=0
Jul 22 02:15:01.956835 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.956793    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2" event={"ID":"ed4159c8-61f5-4665-8a28-e55f0da713c8","Type":"ContainerDied","Data":"612cfe588c527862cc2fbe20f86e2aa526d1b6ef6a7babd3ec63bbc3351364cf"}
Jul 22 02:15:01.958226 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.958207    2560 generic.go:356] "Generic (PLEG): container finished" podID="c27222cf-4271-4347-9efc-e1c58d08a1a7" containerID="03af4ce0433f093d15bf781260f387df840de50516ceebb9be1108493f8ee99a" exitCode=0
Jul 22 02:15:01.958304 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:01.958242    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx" event={"ID":"c27222cf-4271-4347-9efc-e1c58d08a1a7","Type":"ContainerDied","Data":"03af4ce0433f093d15bf781260f387df840de50516ceebb9be1108493f8ee99a"}
Jul 22 02:15:03.102203 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.102177    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"
Jul 22 02:15:03.144309 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.144290    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"
Jul 22 02:15:03.147594 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.147575    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"
Jul 22 02:15:03.970240 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.970203    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2"
Jul 22 02:15:03.970240 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.970213    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744775-5dkf2" event={"ID":"ed4159c8-61f5-4665-8a28-e55f0da713c8","Type":"ContainerDied","Data":"eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57"}
Jul 22 02:15:03.970543 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.970271    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaeaef480fdfebe56d5f8b22e6ab025dd259c98b2d9466cfd7ac91da28938f57"
Jul 22 02:15:03.971646 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.971627    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"
Jul 22 02:15:03.971646 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.971631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx" event={"ID":"c27222cf-4271-4347-9efc-e1c58d08a1a7","Type":"ContainerDied","Data":"ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef"}
Jul 22 02:15:03.971845 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.971682    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac2a7c821eff23349df74fbae1c4969eee942333859a3d91533ffd37890a0cef"
Jul 22 02:15:03.973222 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.973204    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"
Jul 22 02:15:03.973222 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.973210    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg" event={"ID":"e5d1b8f4-8cc3-46e8-a0d5-becb07d07395","Type":"ContainerDied","Data":"da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b"}
Jul 22 02:15:03.973432 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:03.973250    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da2546da4508399a95d1bf13b53610aa8d6b7de57b57766d761e381089aa930b"
Jul 22 02:15:06.795640 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:06.795609    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"]
Jul 22 02:15:06.801000 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:06.800973    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-40b699-29744775-dvnwx"]
Jul 22 02:15:07.991548 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:07.991509    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c27222cf-4271-4347-9efc-e1c58d08a1a7" path="/var/lib/kubelet/pods/c27222cf-4271-4347-9efc-e1c58d08a1a7/volumes"
Jul 22 02:15:11.386233 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:11.386190    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"]
Jul 22 02:15:11.389113 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:11.389083    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-40b699-pnkhm"]
Jul 22 02:15:11.997410 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:11.997367    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0957d9c-3591-4b02-86b2-04dce1b1f210" path="/var/lib/kubelet/pods/f0957d9c-3591-4b02-86b2-04dce1b1f210/volumes"
Jul 22 02:15:18.268827 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.268795    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:15:18.269230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.269065    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy" containerID="cri-o://83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e" gracePeriod=30
Jul 22 02:15:18.666456 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666422    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"]
Jul 22 02:15:18.666768 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666755    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4159c8-61f5-4665-8a28-e55f0da713c8" containerName="cleanup"
Jul 22 02:15:18.666815 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666770    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4159c8-61f5-4665-8a28-e55f0da713c8" containerName="cleanup"
Jul 22 02:15:18.666815 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666783    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5d1b8f4-8cc3-46e8-a0d5-becb07d07395" containerName="cleanup"
Jul 22 02:15:18.666815 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666788    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5d1b8f4-8cc3-46e8-a0d5-becb07d07395" containerName="cleanup"
Jul 22 02:15:18.666815 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666801    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27222cf-4271-4347-9efc-e1c58d08a1a7" containerName="cleanup"
Jul 22 02:15:18.666815 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666807    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27222cf-4271-4347-9efc-e1c58d08a1a7" containerName="cleanup"
Jul 22 02:15:18.666970 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666866    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4159c8-61f5-4665-8a28-e55f0da713c8" containerName="cleanup"
Jul 22 02:15:18.666970 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666874    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c27222cf-4271-4347-9efc-e1c58d08a1a7" containerName="cleanup"
Jul 22 02:15:18.666970 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.666882    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5d1b8f4-8cc3-46e8-a0d5-becb07d07395" containerName="cleanup"
Jul 22 02:15:18.669828 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.669812    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:18.673582 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.673561    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-twths\""
Jul 22 02:15:18.678872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.678848    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"]
Jul 22 02:15:18.757202 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.757180    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4\" (UID: \"3a427657-e42b-48b6-9e53-4c2ae4eab41a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:18.858580 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.858534    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4\" (UID: \"3a427657-e42b-48b6-9e53-4c2ae4eab41a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:18.859121 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.859100    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4\" (UID: \"3a427657-e42b-48b6-9e53-4c2ae4eab41a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:18.980557 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:18.980500    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:19.112005 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:19.111979    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"]
Jul 22 02:15:19.113722 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:15:19.113692    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a427657_e42b_48b6_9e53_4c2ae4eab41a.slice/crio-ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9 WatchSource:0}: Error finding container ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9: Status 404 returned error can't find the container with id ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9
Jul 22 02:15:20.057736 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:20.057677    2560 generic.go:356] "Generic (PLEG): container finished" podID="3a427657-e42b-48b6-9e53-4c2ae4eab41a" containerID="8eb7a57d0a71a01ed088041ae07c6872f755d9ddfb204942cf1e02f2cb89f22f" exitCode=0
Jul 22 02:15:20.057736 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:20.057727    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4" event={"ID":"3a427657-e42b-48b6-9e53-4c2ae4eab41a","Type":"ContainerDied","Data":"8eb7a57d0a71a01ed088041ae07c6872f755d9ddfb204942cf1e02f2cb89f22f"}
Jul 22 02:15:20.058050 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:20.057760    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4" event={"ID":"3a427657-e42b-48b6-9e53-4c2ae4eab41a","Type":"ContainerStarted","Data":"ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9"}
Jul 22 02:15:21.201342 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:21.201319    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:21.278576 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:21.278551    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca\") pod \"3a427657-e42b-48b6-9e53-4c2ae4eab41a\" (UID: \"3a427657-e42b-48b6-9e53-4c2ae4eab41a\") "
Jul 22 02:15:21.278860 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:21.278841    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a427657-e42b-48b6-9e53-4c2ae4eab41a" (UID: "3a427657-e42b-48b6-9e53-4c2ae4eab41a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:15:21.380059 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:21.380035    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a427657-e42b-48b6-9e53-4c2ae4eab41a-maas-api-service-ca\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:22.069082 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:22.069057    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"
Jul 22 02:15:22.069198 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:22.069079    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4" event={"ID":"3a427657-e42b-48b6-9e53-4c2ae4eab41a","Type":"ContainerDied","Data":"ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9"}
Jul 22 02:15:22.069198 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:22.069118    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae934119a41361db5ef8e7d25b1d5272fe1413e917896e3742dcdc9f7d060ce9"
Jul 22 02:15:23.514173 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.514152    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:15:23.596578 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596523    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596578 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596554    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596578 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596576    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596757 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596601    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596757 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596625    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596757 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596677    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.596757 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596734    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.597009 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596762    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwkfm\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.597009 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596810    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket\") pod \"dd531303-7561-4604-a234-d5b2f2d53a75\" (UID: \"dd531303-7561-4604-a234-d5b2f2d53a75\") "
Jul 22 02:15:23.597009 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596826    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:15:23.597009 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.596846    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:15:23.597230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.597026    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 02:15:23.597230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.597071    2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.597230 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.597096    2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-workload-certs\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.597409 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.597289    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:15:23.597604 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.597578    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:15:23.598783 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.598760    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:15:23.598892 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.598870    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 02:15:23.598948 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.598882    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm" (OuterVolumeSpecName: "kube-api-access-gwkfm") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "kube-api-access-gwkfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 02:15:23.598948 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.598875    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd531303-7561-4604-a234-d5b2f2d53a75" (UID: "dd531303-7561-4604-a234-d5b2f2d53a75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 02:15:23.697891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697864    2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-data\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.697891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697888    2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd531303-7561-4604-a234-d5b2f2d53a75-istiod-ca-cert\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.698032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697897    2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-istio-token\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.698032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697904    2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd531303-7561-4604-a234-d5b2f2d53a75-istio-podinfo\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.698032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697912    2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-istio-envoy\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.698032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697921    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwkfm\" (UniqueName: \"kubernetes.io/projected/dd531303-7561-4604-a234-d5b2f2d53a75-kube-api-access-gwkfm\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:23.698032 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:23.697929    2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd531303-7561-4604-a234-d5b2f2d53a75-credential-socket\") on node \"ip-10-0-140-155.ec2.internal\" DevicePath \"\""
Jul 22 02:15:24.079878 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.079848    2560 generic.go:356] "Generic (PLEG): container finished" podID="dd531303-7561-4604-a234-d5b2f2d53a75" containerID="83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e" exitCode=0
Jul 22 02:15:24.079975 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.079884    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"
Jul 22 02:15:24.079975 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.079921    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerDied","Data":"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"}
Jul 22 02:15:24.079975 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.079971    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj" event={"ID":"dd531303-7561-4604-a234-d5b2f2d53a75","Type":"ContainerDied","Data":"60181c5a9f5058244d5b7a990d9ae1c83bda96a85edca4357a0f96996b821ce5"}
Jul 22 02:15:24.080084 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.079995    2560 scope.go:117] "RemoveContainer" containerID="83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"
Jul 22 02:15:24.088523 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.088496    2560 scope.go:117] "RemoveContainer" containerID="3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"
Jul 22 02:15:24.095937 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.095923    2560 scope.go:117] "RemoveContainer" containerID="83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"
Jul 22 02:15:24.096187 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:15:24.096166    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e\": container with ID starting with 83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e not found: ID does not exist" containerID="83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"
Jul 22 02:15:24.096233 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.096195    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e"} err="failed to get container status \"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e\": rpc error: code = NotFound desc = could not find container \"83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e\": container with ID starting with 83b6addbc2e1b10516e127b24534285cfde8d6fb67d795e7fcacb3f642e46f4e not found: ID does not exist"
Jul 22 02:15:24.096233 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.096211    2560 scope.go:117] "RemoveContainer" containerID="3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"
Jul 22 02:15:24.096455 ip-10-0-140-155 kubenswrapper[2560]: E0722 02:15:24.096438    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55\": container with ID starting with 3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55 not found: ID does not exist" containerID="3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"
Jul 22 02:15:24.096512 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.096461    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55"} err="failed to get container status \"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55\": rpc error: code = NotFound desc = could not find container \"3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55\": container with ID starting with 3e7a2b141e77d91f46b598aa7cc42bfe3bece108fc51ff601437d68416050d55 not found: ID does not exist"
Jul 22 02:15:24.099275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.099248    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:15:24.103748 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:24.103730    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-40b699-openshift-default-7fd444455f-fj2pj"]
Jul 22 02:15:25.991828 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:25.991791    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" path="/var/lib/kubelet/pods/dd531303-7561-4604-a234-d5b2f2d53a75/volumes"
Jul 22 02:15:29.112807 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:29.112766    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"]
Jul 22 02:15:29.115749 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:29.115724    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1f4af7-29744775-lmtxg"]
Jul 22 02:15:29.991468 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:29.991433    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5d1b8f4-8cc3-46e8-a0d5-becb07d07395" path="/var/lib/kubelet/pods/e5d1b8f4-8cc3-46e8-a0d5-becb07d07395/volumes"
Jul 22 02:15:33.738607 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:33.738566    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"]
Jul 22 02:15:33.741982 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:33.741956    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-1f4af7-q9gx4"]
Jul 22 02:15:33.990299 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:33.990228    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a427657-e42b-48b6-9e53-4c2ae4eab41a" path="/var/lib/kubelet/pods/3a427657-e42b-48b6-9e53-4c2ae4eab41a/volumes"
Jul 22 02:15:47.623436 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:47.623390    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hg6hf_b3721b4d-0d8f-42da-b84c-751b3d094475/manager/0.log"
Jul 22 02:15:47.869656 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:47.869628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9869b8c7c-nppzp_b36749e4-1826-418f-bf79-ebba86f6418e/copy-manifests/0.log"
Jul 22 02:15:47.877507 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:47.877442    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9869b8c7c-nppzp_b36749e4-1826-418f-bf79-ebba86f6418e/manager/0.log"
Jul 22 02:15:48.531746 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:48.531713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744745-2mh6w_a0c11ab0-6bb7-479e-acde-c59d52e492bf/cleanup/0.log"
Jul 22 02:15:48.662247 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:48.662194    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744760-ssvkg_6d00e6d6-351e-482e-bbe7-d060660045d6/cleanup/0.log"
Jul 22 02:15:48.781992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:48.781901    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744775-5dkf2_ed4159c8-61f5-4665-8a28-e55f0da713c8/cleanup/0.log"
Jul 22 02:15:48.904941 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:48.904914    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-fjdlg_6529147f-5bbf-4344-b6f7-794a610329c6/postgres/0.log"
Jul 22 02:15:50.594204 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:50.594174    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gd7sm_cb9818f7-8f7a-4963-b148-1fc73d3bbb7f/manager/0.log"
Jul 22 02:15:50.727102 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:50.727079    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9vb8n_e023244e-57b5-4967-83d0-2f3728285350/kuadrant-console-plugin/0.log"
Jul 22 02:15:51.879272 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:51.879250    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7dbc5fd-98jlr_aedf98a0-caa4-4d30-acbc-466eed9bdc84/kube-auth-proxy/0.log"
Jul 22 02:15:52.398001 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:52.397959    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-bxr8b_b467903e-6246-41df-81ea-9dec308b66fb/payload-processing/0.log"
Jul 22 02:15:52.902843 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:52.902784    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-52hhv_25478eee-88d1-4d56-b9b6-0b082d411c26/main/0.log"
Jul 22 02:15:53.157517 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:53.157459    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vb74w_0f83dfe2-b843-41b9-ba57-50d268932db4/main/0.log"
Jul 22 02:15:59.796382 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796299    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bvctt/must-gather-nl58l"]
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796667    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796679    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796689    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796695    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796701    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a427657-e42b-48b6-9e53-4c2ae4eab41a" containerName="revoke-keys"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796706    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a427657-e42b-48b6-9e53-4c2ae4eab41a" containerName="revoke-keys"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796768    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.796872 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796777    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a427657-e42b-48b6-9e53-4c2ae4eab41a" containerName="revoke-keys"
Jul 22 02:15:59.797108 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.796883    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd531303-7561-4604-a234-d5b2f2d53a75" containerName="istio-proxy"
Jul 22 02:15:59.800155 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.800136    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:15:59.808142 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.808122    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bvctt\"/\"default-dockercfg-zqgnq\""
Jul 22 02:15:59.809324 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.809302    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bvctt\"/\"openshift-service-ca.crt\""
Jul 22 02:15:59.809324 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.809320    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bvctt\"/\"kube-root-ca.crt\""
Jul 22 02:15:59.830873 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.830847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bvctt/must-gather-nl58l"]
Jul 22 02:15:59.966194 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.966168    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9qc\" (UniqueName: \"kubernetes.io/projected/014a9150-9a22-4b33-aa50-ec67f03cf124-kube-api-access-kx9qc\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:15:59.966293 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:15:59.966205    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/014a9150-9a22-4b33-aa50-ec67f03cf124-must-gather-output\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.067023 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.066951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9qc\" (UniqueName: \"kubernetes.io/projected/014a9150-9a22-4b33-aa50-ec67f03cf124-kube-api-access-kx9qc\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.067023 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.066996    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/014a9150-9a22-4b33-aa50-ec67f03cf124-must-gather-output\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.067347 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.067328    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/014a9150-9a22-4b33-aa50-ec67f03cf124-must-gather-output\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.076412 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.076377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9qc\" (UniqueName: \"kubernetes.io/projected/014a9150-9a22-4b33-aa50-ec67f03cf124-kube-api-access-kx9qc\") pod \"must-gather-nl58l\" (UID: \"014a9150-9a22-4b33-aa50-ec67f03cf124\") " pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.108847 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.108827    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bvctt/must-gather-nl58l"
Jul 22 02:16:00.227886 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.227864    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bvctt/must-gather-nl58l"]
Jul 22 02:16:00.230331 ip-10-0-140-155 kubenswrapper[2560]: W0722 02:16:00.230305    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod014a9150_9a22_4b33_aa50_ec67f03cf124.slice/crio-0afbcad411097dc33ccfe69749d43b83a304ee19b67f0168f475786a077a3a72 WatchSource:0}: Error finding container 0afbcad411097dc33ccfe69749d43b83a304ee19b67f0168f475786a077a3a72: Status 404 returned error can't find the container with id 0afbcad411097dc33ccfe69749d43b83a304ee19b67f0168f475786a077a3a72
Jul 22 02:16:00.248720 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:00.248695    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bvctt/must-gather-nl58l" event={"ID":"014a9150-9a22-4b33-aa50-ec67f03cf124","Type":"ContainerStarted","Data":"0afbcad411097dc33ccfe69749d43b83a304ee19b67f0168f475786a077a3a72"}
Jul 22 02:16:01.258346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:01.258255    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bvctt/must-gather-nl58l" event={"ID":"014a9150-9a22-4b33-aa50-ec67f03cf124","Type":"ContainerStarted","Data":"f7e4f9cc290c95e48f396950687f941d4e3bcd3e502dffcbe822625cabbf998e"}
Jul 22 02:16:01.258346 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:01.258296    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bvctt/must-gather-nl58l" event={"ID":"014a9150-9a22-4b33-aa50-ec67f03cf124","Type":"ContainerStarted","Data":"d79fe1603a71890bcaa929cb7a75baf4b58cc073787da0f3607aa628b07a0bbb"}
Jul 22 02:16:01.274341 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:01.274278    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bvctt/must-gather-nl58l" podStartSLOduration=1.63132674 podStartE2EDuration="2.274257714s" totalImagesPullingTime="642.930974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:16:01.273070239 +0000 UTC m=+2879.821488361" watchObservedRunningTime="2026-07-22 02:16:01.274257714 +0000 UTC m=+2879.822675836"
Jul 22 02:16:02.383609 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:02.383575    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8t8r6_a15a6203-7643-4de5-b179-d107b53699f1/global-pull-secret-syncer/0.log"
Jul 22 02:16:02.577419 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:02.577336    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lh9bc_fc3b75e8-d9b2-42d6-8947-70ea7629fea8/konnectivity-agent/0.log"
Jul 22 02:16:02.682929 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:02.682902    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-155.ec2.internal_3a5bc24e7d34684078d3262a35f51e63/haproxy/0.log"
Jul 22 02:16:07.287371 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:07.287335    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gd7sm_cb9818f7-8f7a-4963-b148-1fc73d3bbb7f/manager/0.log"
Jul 22 02:16:07.323926 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:07.323899    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9vb8n_e023244e-57b5-4967-83d0-2f3728285350/kuadrant-console-plugin/0.log"
Jul 22 02:16:09.562287 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.562255    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56bd4ffb4c-rtqjf_721555e7-f881-4c94-834c-9ace160767b4/metrics-server/0.log"
Jul 22 02:16:09.593925 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.593896    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-jwpm5_f4cb14e4-128d-483f-baac-4b4beb8c8bb3/monitoring-plugin/0.log"
Jul 22 02:16:09.630107 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.630075    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26d4w_e1518b7f-525f-4bc5-b72b-3e83d66e5fd3/node-exporter/0.log"
Jul 22 02:16:09.654688 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.654611    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26d4w_e1518b7f-525f-4bc5-b72b-3e83d66e5fd3/kube-rbac-proxy/0.log"
Jul 22 02:16:09.678810 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.678787    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26d4w_e1518b7f-525f-4bc5-b72b-3e83d66e5fd3/init-textfile/0.log"
Jul 22 02:16:09.883725 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.883694    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6pxfs_d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b/kube-rbac-proxy-main/0.log"
Jul 22 02:16:09.914150 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.914072    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6pxfs_d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b/kube-rbac-proxy-self/0.log"
Jul 22 02:16:09.938532 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.938502    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-6pxfs_d6d61fe2-4555-4c6d-b971-d4fd9ae3d95b/openshift-state-metrics/0.log"
Jul 22 02:16:09.978044 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:09.978017    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/prometheus/0.log"
Jul 22 02:16:10.002002 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.001958    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/config-reloader/0.log"
Jul 22 02:16:10.026504 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.026474    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/thanos-sidecar/0.log"
Jul 22 02:16:10.059988 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.059958    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/kube-rbac-proxy-web/0.log"
Jul 22 02:16:10.081832 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.081809    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/kube-rbac-proxy/0.log"
Jul 22 02:16:10.104822 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.104800    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/kube-rbac-proxy-thanos/0.log"
Jul 22 02:16:10.132296 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.132267    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5516f4a2-0f21-4643-b390-357b8a8f3241/init-config-reloader/0.log"
Jul 22 02:16:10.207054 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:10.206975    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-pdjhc_4251218f-e27c-4e53-a94c-924e59e22d79/prometheus-operator-admission-webhook/0.log"
Jul 22 02:16:11.744635 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.744604    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"]
Jul 22 02:16:11.750129 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.750106    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.760658 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.760635    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"]
Jul 22 02:16:11.779759 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.779731    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-sys\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.779759 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.779762    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-lib-modules\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.779999 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.779810    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-proc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.779999 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.779832    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-podres\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.779999 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.779863    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2rc\" (UniqueName: \"kubernetes.io/projected/352f810e-60db-47ee-8a29-8bb091eb0480-kube-api-access-lp2rc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881131 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881093    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2rc\" (UniqueName: \"kubernetes.io/projected/352f810e-60db-47ee-8a29-8bb091eb0480-kube-api-access-lp2rc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-sys\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881206    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-lib-modules\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881284    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-proc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881306 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881286    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-sys\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881535 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881307    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-podres\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881535 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881431    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-podres\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881535 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881482    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-proc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.881535 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.881509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/352f810e-60db-47ee-8a29-8bb091eb0480-lib-modules\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:11.889521 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:11.889498    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2rc\" (UniqueName: \"kubernetes.io/projected/352f810e-60db-47ee-8a29-8bb091eb0480-kube-api-access-lp2rc\") pod \"perf-node-gather-daemonset-9vvht\" (UID: \"352f810e-60db-47ee-8a29-8bb091eb0480\") " pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:12.062869 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:12.062789    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:12.238302 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:12.237283    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"]
Jul 22 02:16:12.334772 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:12.334746    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht" event={"ID":"352f810e-60db-47ee-8a29-8bb091eb0480","Type":"ContainerStarted","Data":"3bbca3b355910091fcf0b8561fd1872e693f4cb7551611de2317b920bfb6d307"}
Jul 22 02:16:13.344335 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:13.344305    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht" event={"ID":"352f810e-60db-47ee-8a29-8bb091eb0480","Type":"ContainerStarted","Data":"5a4491e805dc195407e19e321ef9031b87732208c994e054345d057ccc02a30d"}
Jul 22 02:16:13.344709 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:13.344388    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:13.364639 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:13.364588    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht" podStartSLOduration=2.36456981 podStartE2EDuration="2.36456981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:16:13.363091351 +0000 UTC m=+2891.911509472" watchObservedRunningTime="2026-07-22 02:16:13.36456981 +0000 UTC m=+2891.912987934"
Jul 22 02:16:14.592948 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:14.592922    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vk6l2_48450b9b-d90c-49ec-b8d6-f975e43a0161/dns/0.log"
Jul 22 02:16:14.620994 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:14.620970    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vk6l2_48450b9b-d90c-49ec-b8d6-f975e43a0161/kube-rbac-proxy/0.log"
Jul 22 02:16:14.670293 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:14.670268    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngqhh_c6ef6ede-818c-4d00-a20a-6d197988f37c/dns-node-resolver/0.log"
Jul 22 02:16:15.243329 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:15.243290    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64c6f57f95-zk9c8_749625cb-3e84-40a2-996d-f0a81a5a4de9/registry/0.log"
Jul 22 02:16:15.275526 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:15.275497    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-b88qd_c43001b2-2152-4107-8421-cf2cab3e7e7b/node-ca/0.log"
Jul 22 02:16:16.479275 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:16.479242    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7dbc5fd-98jlr_aedf98a0-caa4-4d30-acbc-466eed9bdc84/kube-auth-proxy/0.log"
Jul 22 02:16:16.687465 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:16.687435    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-bxr8b_b467903e-6246-41df-81ea-9dec308b66fb/payload-processing/0.log"
Jul 22 02:16:17.482992 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:17.482955    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qc6md_50cb7235-e418-4f82-83ee-ddcdb4514803/serve-healthcheck-canary/0.log"
Jul 22 02:16:18.077027 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:18.076996    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snkdq_f5870761-98cb-4f9c-b6a0-853e2181e674/kube-rbac-proxy/0.log"
Jul 22 02:16:18.103790 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:18.103760    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snkdq_f5870761-98cb-4f9c-b6a0-853e2181e674/exporter/0.log"
Jul 22 02:16:18.131738 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:18.131713    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-snkdq_f5870761-98cb-4f9c-b6a0-853e2181e674/extractor/0.log"
Jul 22 02:16:19.359112 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:19.359083    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bvctt/perf-node-gather-daemonset-9vvht"
Jul 22 02:16:20.844960 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:20.844879    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hg6hf_b3721b4d-0d8f-42da-b84c-751b3d094475/manager/0.log"
Jul 22 02:16:20.921477 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:20.921357    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9869b8c7c-nppzp_b36749e4-1826-418f-bf79-ebba86f6418e/manager/0.log"
Jul 22 02:16:20.943295 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:20.943269    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9869b8c7c-nppzp_b36749e4-1826-418f-bf79-ebba86f6418e/copy-manifests/0.log"
Jul 22 02:16:21.464626 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:21.464556    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7dbc5fd-98jlr_aedf98a0-caa4-4d30-acbc-466eed9bdc84/kube-auth-proxy/0.log"
Jul 22 02:16:21.604525 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:21.604485    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-bxr8b_b467903e-6246-41df-81ea-9dec308b66fb/payload-processing/0.log"
Jul 22 02:16:22.386550 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:22.386518    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7898dc9f59-qbjf6_a0ccc9b1-4473-4013-93bf-05b13150ac86/manager/0.log"
Jul 22 02:16:22.414891 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:22.414863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wpcfc_697861bd-2c07-4fd2-ba03-8355f1f43a73/openshift-lws-operator/0.log"
Jul 22 02:16:22.866351 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:22.866322    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d7dbc5fd-98jlr_aedf98a0-caa4-4d30-acbc-466eed9bdc84/kube-auth-proxy/0.log"
Jul 22 02:16:23.013638 ip-10-0-140-155 kubenswrapper[2560]: I0722 02:16:23.013601    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-bxr8b_b467903e-6246-41df-81ea-9dec308b66fb/payload-processing/0.log"