Jul 10 02:12:20.515242 ip-10-0-140-223 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 02:12:20.515258 ip-10-0-140-223 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 02:12:20.515267 ip-10-0-140-223 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 02:12:20.515577 ip-10-0-140-223 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 02:12:31.801669 ip-10-0-140-223 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 02:12:31.801686 ip-10-0-140-223 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 282ceb1a9ba04f0f9297fe41e69acbb5 --
Jul 10 02:14:34.584990 ip-10-0-140-223 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 02:14:34.993781 ip-10-0-140-223 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:34.993781 ip-10-0-140-223 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 02:14:34.993781 ip-10-0-140-223 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:34.993781 ip-10-0-140-223 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 02:14:34.993781 ip-10-0-140-223 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:34.994455 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:34.993818    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 02:14:34.997939 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997924    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:34.997939 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997940    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997944    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997948    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997951    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997954    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997958    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997961    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997965    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997969    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997973    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997975    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997978    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997982    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997985    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997987    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997990    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997993    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997996    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.997998    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998000    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:34.998033 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998003    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998006    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998008    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998011    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998013    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998016    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998018    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998020    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998023    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998025    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998028    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998030    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998033    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998037    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998041    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998045    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998048    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998052    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998055    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:34.998509 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998058    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998060    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998063    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998066    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998068    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998071    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998073    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998076    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998078    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998081    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998083    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998085    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998088    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998090    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998093    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998097    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998100    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998103    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998105    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998108    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:34.998980 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998110    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998113    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998115    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998117    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998120    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998123    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998125    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998127    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998130    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998132    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998135    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998138    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998141    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998143    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998146    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998148    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998150    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998153    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998155    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998157    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:34.999503 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998160    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998162    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998165    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998168    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998170    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998172    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998175    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998178    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998181    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998184    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998187    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998189    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998192    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998194    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998197    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998199    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998202    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998206    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.998209    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.000006 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999838    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999846    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999850    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999852    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999855    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999858    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999861    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999863    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999866    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999868    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999871    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999875    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999877    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999880    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999882    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999887    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999890    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999893    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999896    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999899    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:35.000480 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999901    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999904    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999907    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999909    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999912    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999914    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999917    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999920    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999922    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999925    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999927    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999929    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999932    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999935    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999938    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999940    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999942    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999945    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999949    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999951    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:35.000997 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999954    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999957    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999959    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999963    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999965    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999968    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999970    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999973    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999975    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999978    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999980    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999982    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999985    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999987    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999990    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999995    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:34.999999    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000002    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000004    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:35.001491 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000007    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000010    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000013    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000016    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000019    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000021    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000024    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000026    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000029    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000032    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000034    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000036    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000039    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000042    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000044    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000047    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000050    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000052    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000054    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:35.001982 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000057    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000059    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000062    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000064    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000067    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000069    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000072    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000076    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000079    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000082    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000084    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000087    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000089    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000092    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000094    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000096    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000099    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000101    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000104    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000106    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:35.002462 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000108    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000484    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000492    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000496    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000499    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000502    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000505    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000507    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000510    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000513    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000515    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000518    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000520    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000523    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000525    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000528    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000530    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000533    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000536    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000538    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:35.002958 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000541    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000543    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000546    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000549    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000551    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000554    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000556    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000559    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000561    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000564    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000567    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000569    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000572    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000574    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000576    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000579    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000581    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000583    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000586    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:35.003465 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000588    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000592    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000594    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000597    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000599    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000601    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000604    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000606    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000610    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000613    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000615    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000618    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000621    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000623    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000626    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000628    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000631    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000633    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000636    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000639    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:35.003966 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000641    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000643    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000646    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000648    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000650    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000653    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000655    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000658    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000660    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000662    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000665    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000667    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000671    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000675    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000678    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000681    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000683    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000687    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000689    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000692    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:35.004453 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000694    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000697    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000699    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000703    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000707    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000710    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000713    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000715    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000718    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000721    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000724    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000742    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000745    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000748    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000750    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000753    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000755    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000758    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000760    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000763    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:35.004995 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.000765    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000833    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000844    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000850    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000854    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000858    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000861    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000865    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000880    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000883    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000886    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000889    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000892    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000895    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000898    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000903    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000905    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000908    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000911    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000915    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000918    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000921    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000924    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000927    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 02:14:35.005482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000930    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000933    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000936    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000940    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000943    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000945    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000948    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000951    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000954    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000957    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000960    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000963    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000966    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000969    2579 flags.go:64] FLAG: --enable-server="true"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000972    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000977    2579 flags.go:64] FLAG: --event-burst="100"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000980    2579 flags.go:64] FLAG: --event-qps="50"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000982    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000985    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000988    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000991    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000994    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.000998    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001002    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001004    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 02:14:35.006126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001007    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001010    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001013    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001016    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001018    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001021    2579 flags.go:64] FLAG: --feature-gates=""
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001025    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001028    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001031    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001034    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001037    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001040    2579 flags.go:64] FLAG: --help="false"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001042    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001045    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001048    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001051    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001054    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001057    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001060    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001063    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001066    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001069    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001072    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001075    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 02:14:35.006761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001078    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001081    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001083    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001086    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001092    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001094    2579 flags.go:64] FLAG: --lock-file=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001098    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001101    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001104    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001109    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001112    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001115    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001117    2579 flags.go:64] FLAG: --logging-format="text"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001120    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001123    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001126    2579 flags.go:64] FLAG: --manifest-url=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001129    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001133    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001136    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001139    2579 flags.go:64] FLAG: --max-pods="110"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001142    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001145    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001148    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001151    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001154    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 02:14:35.007355 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001157    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001159    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001166    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001169    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001172    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001175    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001180    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001186    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001189    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001192    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001195    2579 flags.go:64] FLAG: --port="10250"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001202    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001204    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-018737319241714c0"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001208    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001211    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001214    2579 flags.go:64] FLAG: --register-node="true"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001217    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001221    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001223    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001226    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001229    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001232    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001235    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001238    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001241    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 02:14:35.008011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001244    2579 flags.go:64] FLAG: --runonce="false"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001246    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001249    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001252    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001255    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001257    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001260    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001263    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001266    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001269    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001271    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001274    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001277    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001280    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001283    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001286    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001291    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001294    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001299    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001302    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001305    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001309    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001312    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001315    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001317    2579 flags.go:64] FLAG: --v="2"
Jul 10 02:14:35.008658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001321    2579 flags.go:64] FLAG: --version="false"
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001327    2579 flags.go:64] FLAG: --vmodule=""
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001331    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.001334    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001441    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001444    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001447    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001450    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001453    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001455    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001457    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001460    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001463    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001465    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001468    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001470    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001473    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001475    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001478    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001480    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:35.009313 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001483    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001485    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001488    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001491    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001494    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001498    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001501    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001505    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001509    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001511    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001514    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001517    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001519    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001522    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001524    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001527    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001529    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001532    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001535    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.009863 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001537    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001539    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001542    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001544    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001547    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001549    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001552    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001554    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001557    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001559    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001562    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001564    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001566    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001569    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001571    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001574    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001577    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001579    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001584    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001588    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:35.010332 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001591    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001595    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001599    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001603    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001606    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001609    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001611    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001614    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001616    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001619    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001621    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001623    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001626    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001629    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001632    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001634    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001637    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001639    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001641    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001644    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:35.010848 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001646    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001649    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001651    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001654    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001656    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001658    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001661    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001663    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001665    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001668    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001672    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001675    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001677    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001681    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001683    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001686    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001689    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001691    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001694    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001696    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:35.011390 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001699    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001701    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001704    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.001706    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.002554    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.009619    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.009634    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.010693    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.010700    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010770    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010776    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010779    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010781    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010785    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:35.012042 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010787    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010790    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010793    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010796    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010798    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010801    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010803    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010807    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010809    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010812    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010814    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010817    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010819    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010822    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010824    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010827    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010830    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010832    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010835    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:35.012614 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010837    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010840    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010842    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010845    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010848    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010851    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010854    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010857    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010859    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010862    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010865    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010867    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010870    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010872    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010875    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010878    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010880    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010882    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010885    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010887    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010890    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.013175 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010893    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010896    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010898    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010900    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010903    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010906    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010908    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010910    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010913    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010915    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010918    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010920    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010923    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010925    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010928    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010931    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010933    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010938    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010941    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010944    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.013682 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010947    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010950    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010953    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010955    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010959    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010962    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010964    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010967    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010970    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010972    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010975    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010977    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010980    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010983    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010985    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010988    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010990    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010993    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010995    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.014189 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.010998    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011002    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011006    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011008    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011011    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011013    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011016    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011019    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011021    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011024    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011026    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011029    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011031    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011035    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:35.014711 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011038    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.011044    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011155    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011160    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011163    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011166    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011169    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011172    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011175    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011178    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011180    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011183    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011185    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011188    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:35.015092 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011190    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011193    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011196    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011199    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011202    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011204    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011207    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011210    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011212    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011215    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011217    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011220    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011223    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011225    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011228    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011230    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011233    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011235    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011237    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011240    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:35.015442 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011243    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011245    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011247    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011250    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011252    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011254    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011257    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011259    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011262    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011264    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011267    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011269    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011272    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011274    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011277    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011279    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011282    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011284    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011287    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011289    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:35.015950 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011291    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011294    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011296    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011299    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011302    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011304    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011307    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011309    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011311    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011314    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011317    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011320    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011323    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011325    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011328    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011330    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011332    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011335    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011337    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011340    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:35.016550 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011342    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011344    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011347    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011349    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011352    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011356    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011359    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011362    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011365    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011368    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011370    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011373    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011376    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011380    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011382    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011385    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011387    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011390    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011392    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:35.017071 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011395    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011397    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011400    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011403    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011405    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011409    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011412    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:35.011415    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.011419    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.011514    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.014341    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.016036    2579 server.go:1019] "Starting client certificate rotation"
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.016144    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:14:35.017611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.016198    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:14:35.018907 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.018891    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:14:35.020085 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.020062    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:14:35.032294 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.032280    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 10 02:14:35.034333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.034317    2579 log.go:25] "Validated CRI v1 image API"
Jul 10 02:14:35.035906 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.035884    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 02:14:35.039399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.039375    2579 fs.go:135] Filesystem UUIDs: map[2942df30-6dc2-4595-8cdf-f495abe5774b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8da9c465-08dc-448c-b66b-88698180bbcb:/dev/nvme0n1p4]
Jul 10 02:14:35.039466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.039398    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 02:14:35.044057 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.044040    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:14:35.045032 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.044929    2579 manager.go:216] Machine: {Timestamp:2026-07-10 02:14:35.043691503 +0000 UTC m=+0.340273269 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093998 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f23d4808f9a56f70d306816e94782 SystemUUID:ec2f23d4-808f-9a56-f70d-306816e94782 BootID:282ceb1a-9ba0-4f0f-9297-fe41e69acbb5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:d2:32:2c:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:d2:32:2c:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:b2:c1:bd:ef:9f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 02:14:35.045032 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.045030    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 02:14:35.045130 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.045101    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 02:14:35.046185 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.046163    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 02:14:35.046310 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.046190    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 02:14:35.047560 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.047549    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 02:14:35.047604 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.047563    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 02:14:35.047604 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.047571    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:14:35.048200 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.048190    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:14:35.049116 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.049106    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 02:14:35.052015 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052005    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:14:35.052277 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052268    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 02:14:35.052307 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052299    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 02:14:35.052337 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052313    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 02:14:35.052337 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052330    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 02:14:35.052393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052340    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 10 02:14:35.052393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052349    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 02:14:35.052771 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052752    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:14:35.052858 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.052789    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:14:35.055621 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.055602    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 02:14:35.057792 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.057775    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 02:14:35.057873 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.057807    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 02:14:35.058458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058447    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 02:14:35.058496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058463    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 02:14:35.058496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058469    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 02:14:35.058496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058474    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 02:14:35.058496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058483    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 02:14:35.058496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058489    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058499    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058508    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058518    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058524    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058533    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 02:14:35.058626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.058541    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 02:14:35.059282 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.059270    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 02:14:35.059316 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.059284    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 02:14:35.062667 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.062652    2579 server.go:1268] "Started kubelet"
Jul 10 02:14:35.062776 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.062751    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 02:14:35.062836 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.062760    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 02:14:35.062885 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.062821    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 02:14:35.063207 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.063129    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 02:14:35.063207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.063138    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 02:14:35.063207 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.063198    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 02:14:35.063541 ip-10-0-140-223 systemd[1]: Started Kubernetes Kubelet.
Jul 10 02:14:35.064656 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.064625    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 02:14:35.065504 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.065488    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 02:14:35.067601 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.067580    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sdq4k"
Jul 10 02:14:35.068079 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.068064    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 02:14:35.070360 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.069410    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-223.ec2.internal.18c0cbede5d94e51  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-223.ec2.internal,UID:ip-10-0-140-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-223.ec2.internal,},FirstTimestamp:2026-07-10 02:14:35.062627921 +0000 UTC m=+0.359209693,LastTimestamp:2026-07-10 02:14:35.062627921 +0000 UTC m=+0.359209693,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-223.ec2.internal,}"
Jul 10 02:14:35.070422 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.070386    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 02:14:35.070472 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.070451    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 02:14:35.070472 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.070465    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.071097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.071078    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 02:14:35.071097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.071091    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 02:14:35.071402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.071107    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 02:14:35.071402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.071207    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 02:14:35.071402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.071211    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 02:14:35.072397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.072379    2579 factory.go:55] Registering systemd factory
Jul 10 02:14:35.072397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.072400    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 10 02:14:35.074012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.073997    2579 factory.go:153] Registering CRI-O factory
Jul 10 02:14:35.074082 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.074015    2579 factory.go:223] Registration of the crio container factory successfully
Jul 10 02:14:35.074082 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.074058    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 02:14:35.074172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.074083    2579 factory.go:103] Registering Raw factory
Jul 10 02:14:35.074172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.074098    2579 manager.go:1196] Started watching for new ooms in manager
Jul 10 02:14:35.074513 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.074493    2579 manager.go:349] Starting recovery of all containers
Jul 10 02:14:35.076353 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.076331    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sdq4k"
Jul 10 02:14:35.078812 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.078785    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 02:14:35.078902 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.078883    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 02:14:35.086655 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.086541    2579 manager.go:354] Recovery completed
Jul 10 02:14:35.090579 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.090567    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.092329 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092311    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.092412 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092341    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.092412 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092356    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.092902 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092885    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 02:14:35.092902 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092903    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 02:14:35.092999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092917    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:14:35.092999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.092893    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.093594 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.093577    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.093661 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.093608    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.093661 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.093624    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.094042 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.094032    2579 policy_none.go:49] "None policy: Start"
Jul 10 02:14:35.094081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.094047    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 02:14:35.094081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.094057    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 02:14:35.094410 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.094328    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-223.ec2.internal.18c0cbede79e762a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-223.ec2.internal,UID:ip-10-0-140-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-223.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-223.ec2.internal,},FirstTimestamp:2026-07-10 02:14:35.09232593 +0000 UTC m=+0.388907695,LastTimestamp:2026-07-10 02:14:35.09232593 +0000 UTC m=+0.388907695,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-223.ec2.internal,}"
Jul 10 02:14:35.094975 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.094961    2579 policy_none.go:47] "Start"
Jul 10 02:14:35.139299 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139284    2579 manager.go:337] "Starting Device Plugin manager"
Jul 10 02:14:35.139387 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.139315    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 02:14:35.139387 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139324    2579 server.go:87] "Starting device plugin registration server"
Jul 10 02:14:35.139576 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139563    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 02:14:35.139632 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139578    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 02:14:35.139677 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139662    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 02:14:35.139775 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139759    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 02:14:35.139775 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.139772    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 02:14:35.140265 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.140248    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 02:14:35.140344 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.140302    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.236921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.236887    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 02:14:35.238234 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.238221    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 02:14:35.238308 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.238255    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 02:14:35.238308 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.238276    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 02:14:35.238408 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.238307    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 02:14:35.239663 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.239648    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.240708 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.240689    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.240825 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.240746    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.240825 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.240780    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.240825 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.240811    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.241002 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.240887    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:35.248546 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.248508    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.248546 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.248530    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-223.ec2.internal\": node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.264682 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.264664    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.339298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.339268    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"]
Jul 10 02:14:35.339449 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.339358    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.340517 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.340502    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.340581 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.340532    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.340581 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.340541    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.342132 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342120    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.342274 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342261    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.342316 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342290    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.342945 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342925    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.342945 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342948    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.343075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.342961    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.343075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.343024    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.343075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.343043    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.343075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.343058    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.344368 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.344352    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.344420 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.344381    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:35.345047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.345032    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:35.345047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.345055    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:35.345176 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.345068    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:35.364894 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.364876    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.372847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.372833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.372900 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.372855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.372900 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.372878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.384385 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.384366    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-223.ec2.internal\" not found" node="ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.389013 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.388999    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-223.ec2.internal\" not found" node="ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.465713 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.465677    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.473043 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.473098 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.473098 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.473159 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.473159 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.473159 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.473137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.566512 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.566442    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.667006 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.666960    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.686462 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.686442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.691631 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:35.691609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:35.767639 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.767607    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.868242 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.868158    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:35.968694 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:35.968657    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:36.015275 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.015254    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 02:14:36.015846 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.015376    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 02:14:36.069500 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:36.069474    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:36.080614 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.080582    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:09:35 +0000 UTC" deadline="2028-01-10 22:32:44.246844519 +0000 UTC"
Jul 10 02:14:36.080614 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.080609    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13196h18m8.166240109s"
Jul 10 02:14:36.169861 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:36.169836    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:36.187856 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.187830    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:36.225344 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:36.225303    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a2c0a00cdbe9d77d0bff3f91f1c3e7.slice/crio-86a148d57bfec4e242195630a60f0f40dd52b9690eaff19cd5e43d479750ab65 WatchSource:0}: Error finding container 86a148d57bfec4e242195630a60f0f40dd52b9690eaff19cd5e43d479750ab65: Status 404 returned error can't find the container with id 86a148d57bfec4e242195630a60f0f40dd52b9690eaff19cd5e43d479750ab65
Jul 10 02:14:36.225695 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:36.225668    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281d1b3a45b212ba9681d002edffe820.slice/crio-9b2a58a6cd9f4b33d66f8d782d01a1510e382cec4003a0c61a949a7e1afd4766 WatchSource:0}: Error finding container 9b2a58a6cd9f4b33d66f8d782d01a1510e382cec4003a0c61a949a7e1afd4766: Status 404 returned error can't find the container with id 9b2a58a6cd9f4b33d66f8d782d01a1510e382cec4003a0c61a949a7e1afd4766
Jul 10 02:14:36.229912 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.229896    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:14:36.242100 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.242064    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerStarted","Data":"9b2a58a6cd9f4b33d66f8d782d01a1510e382cec4003a0c61a949a7e1afd4766"}
Jul 10 02:14:36.243304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.243283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" event={"ID":"16a2c0a00cdbe9d77d0bff3f91f1c3e7","Type":"ContainerStarted","Data":"86a148d57bfec4e242195630a60f0f40dd52b9690eaff19cd5e43d479750ab65"}
Jul 10 02:14:36.270889 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:36.270860    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found"
Jul 10 02:14:36.333457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.333435    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:36.334476 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.334459    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:36.371466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.371435    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:36.389698 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.389634    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:14:36.390783 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.390770    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"
Jul 10 02:14:36.399197 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:36.399184    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:14:37.052653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.052623    2579 apiserver.go:52] "Watching apiserver"
Jul 10 02:14:37.061890 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.061860    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 02:14:37.062382 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.062353    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-c5g6r","kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal","openshift-dns/node-resolver-bbdlw","openshift-image-registry/node-ca-ds69v","openshift-multus/multus-drrlv","openshift-network-operator/iptables-alerter-xj2t7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4","openshift-cluster-node-tuning-operator/tuned-fmmvm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal","openshift-multus/multus-additional-cni-plugins-kg4rd","openshift-multus/network-metrics-daemon-7mx5l","openshift-network-diagnostics/network-check-target-pkhrv","openshift-ovn-kubernetes/ovnkube-node-qhttr","kube-system/global-pull-secret-syncer-f5w9n"]
Jul 10 02:14:37.064240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.064219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.065329 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.065309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.067008 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.066986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.067623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.067598    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.067728 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.067649    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.068129 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.068108    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.068223 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.068134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hc488\""
Jul 10 02:14:37.068379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.068357    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.068715 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.068646    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6nrd7\""
Jul 10 02:14:37.068822 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.068781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 02:14:37.069080 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.069065    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.069541 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.069523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.070566 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.070547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 02:14:37.070779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.070760    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.071075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5gp5x\""
Jul 10 02:14:37.071207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071190    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 02:14:37.071275 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071231    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.071641 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071623    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.071874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071854    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 02:14:37.071999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.071933    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s5ghr\""
Jul 10 02:14:37.072069 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.072145 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072071    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.072223 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-thlhx\""
Jul 10 02:14:37.072283 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072265    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 02:14:37.072346 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072327    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.072694 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072674    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.072794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072713    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 02:14:37.072959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.072942    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.073654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.073634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.075207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.075175    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ft76h\""
Jul 10 02:14:37.075467 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.075451    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 02:14:37.075561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.075475    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 02:14:37.075561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.075476    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzzf9\""
Jul 10 02:14:37.075775 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.075754    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.075895 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.075863    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.076520    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.076565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.076565    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.076655    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rj5fp\""
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.076865    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.077696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:37.078582 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.077815    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:37.080105 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.080083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.080619 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.080598    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.080869 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.080830    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:37.082456 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071228ff-2484-40ba-84cd-49677845886d-hosts-file\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.082553 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.082553 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-system-cni-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.082553 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-cnibin\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.082682 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-bin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.082682 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.082682 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-conf\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.083009 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.082973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-run\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.083092 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.083039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b311cbf3-5339-4afc-ba92-986ba83cde95-host-slash\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.090244 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.083076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.090341 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.090341 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d059ee8-9a60-4470-9110-1bf7ea3e0359-host\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.090457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-var-lib-kubelet\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.090457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-os-release\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.090457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-kubernetes\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.090629 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090475    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t72gz\" (UniqueName: \"kubernetes.io/projected/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kube-api-access-t72gz\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.090629 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4vz\" (UniqueName: \"kubernetes.io/projected/0d059ee8-9a60-4470-9110-1bf7ea3e0359-kube-api-access-hj4vz\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.090629 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-socket-dir-parent\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.090777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-multus\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.090777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090694    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.090777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-k8s-cni-cncf-io\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.090960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-systemd\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.090960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-lib-modules\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.090960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-etc-kubernetes\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.090960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cnibin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.091186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-hostroot\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.091186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.090999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-daemon-config\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.091186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2mzt\" (UniqueName: \"kubernetes.io/projected/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-kube-api-access-b2mzt\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.091186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysconfig\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.091186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091156    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 02:14:37.091725 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvzr7\" (UniqueName: \"kubernetes.io/projected/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-kube-api-access-mvzr7\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.091847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7w98\" (UniqueName: \"kubernetes.io/projected/071228ff-2484-40ba-84cd-49677845886d-kube-api-access-c7w98\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.091847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091789    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 02:14:37.091847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rvjx\" (UniqueName: \"kubernetes.io/projected/b311cbf3-5339-4afc-ba92-986ba83cde95-kube-api-access-6rvjx\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.091847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-system-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071228ff-2484-40ba-84cd-49677845886d-tmp-dir\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.092086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.091980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afafc254-8fa2-4876-9789-425e1c2a8523-konnectivity-ca\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.092086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092027    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-kubelet\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-tuned\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.092304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092094    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 02:14:37.092304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-tmp\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.092304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afafc254-8fa2-4876-9789-425e1c2a8523-agent-certs\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.092304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-os-release\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.092523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cni-binary-copy\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.092523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.092523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b311cbf3-5339-4afc-ba92-986ba83cde95-iptables-alerter-script\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.092701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-device-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.092701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-conf-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.092701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 02:14:37.092937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-netns\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.092937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-modprobe-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.092937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-host\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.092937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.092937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjd66\" (UniqueName: \"kubernetes.io/projected/88c33e16-a136-4dae-a73f-73901b742b10-kube-api-access-wjd66\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.093195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092969    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 02:14:37.093195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.092971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.093195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.093029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d059ee8-9a60-4470-9110-1bf7ea3e0359-serviceca\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.093195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.093080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-multus-certs\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.093195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.093109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-sys\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.093444 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.093235    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 02:14:37.093744 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.093726    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bwtwv\""
Jul 10 02:14:37.094400 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.094378    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:14:37.112405 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.112372    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vv4zj"
Jul 10 02:14:37.122908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.122861    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vv4zj"
Jul 10 02:14:37.172880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.172858    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 02:14:37.193533 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cnibin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-hostroot\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-daemon-config\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2mzt\" (UniqueName: \"kubernetes.io/projected/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-kube-api-access-b2mzt\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysconfig\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvzr7\" (UniqueName: \"kubernetes.io/projected/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-kube-api-access-mvzr7\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.193713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7w98\" (UniqueName: \"kubernetes.io/projected/071228ff-2484-40ba-84cd-49677845886d-kube-api-access-c7w98\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rvjx\" (UniqueName: \"kubernetes.io/projected/b311cbf3-5339-4afc-ba92-986ba83cde95-kube-api-access-6rvjx\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-system-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193789    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071228ff-2484-40ba-84cd-49677845886d-tmp-dir\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afafc254-8fa2-4876-9789-425e1c2a8523-konnectivity-ca\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-kubelet\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-tuned\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-tmp\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-systemd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afafc254-8fa2-4876-9789-425e1c2a8523-agent-certs\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-os-release\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-script-lib\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.194013 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.193987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cni-binary-copy\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-netns\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b311cbf3-5339-4afc-ba92-986ba83cde95-iptables-alerter-script\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-device-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-conf-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194285    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-kubelet-config\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-netns\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-modprobe-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194389    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-host\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjd66\" (UniqueName: \"kubernetes.io/projected/88c33e16-a136-4dae-a73f-73901b742b10-kube-api-access-wjd66\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.194535 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d059ee8-9a60-4470-9110-1bf7ea3e0359-serviceca\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-multus-certs\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-sys\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-env-overrides\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071228ff-2484-40ba-84cd-49677845886d-hosts-file\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-netns\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.195344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.195006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-registration-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.195742 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.195597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-binary-copy\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.195742 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.195691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cnibin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.195885 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.195852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-modprobe-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.195944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.195927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-host\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-multus-certs\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-os-release\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.194889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-dbus\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-system-cni-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-cnibin\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-slash\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-etc-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/071228ff-2484-40ba-84cd-49677845886d-tmp-dir\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-log-socket\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-bin\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196369    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-bin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-hostroot\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-conf\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-sys\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0d059ee8-9a60-4470-9110-1bf7ea3e0359-serviceca\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-bin\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88c33e16-a136-4dae-a73f-73901b742b10-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/071228ff-2484-40ba-84cd-49677845886d-hosts-file\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-d\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-conf-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-run\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-config\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b311cbf3-5339-4afc-ba92-986ba83cde95-host-slash\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-kubelet\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.199981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d059ee8-9a60-4470-9110-1bf7ea3e0359-host\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-var-lib-kubelet\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-os-release\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196984    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-daemon-config\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.196986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-systemd-units\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-kubernetes\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-var-lib-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-netd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovn-node-metrics-cert\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6gr2\" (UniqueName: \"kubernetes.io/projected/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-kube-api-access-t6gr2\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t72gz\" (UniqueName: \"kubernetes.io/projected/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kube-api-access-t72gz\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4vz\" (UniqueName: \"kubernetes.io/projected/0d059ee8-9a60-4470-9110-1bf7ea3e0359-kube-api-access-hj4vz\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-socket-dir-parent\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-multus\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.200752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz7pd\" (UniqueName: \"kubernetes.io/projected/88c8b447-df78-4fb5-9932-c7154765ee06-kube-api-access-vz7pd\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-ovn\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/afafc254-8fa2-4876-9789-425e1c2a8523-konnectivity-ca\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b311cbf3-5339-4afc-ba92-986ba83cde95-iptables-alerter-script\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-k8s-cni-cncf-io\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-systemd\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-lib-modules\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysconfig\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-etc-kubernetes\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-kubelet\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-node-log\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-system-cni-dir\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-device-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-system-cni-dir\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-cnibin\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-kubernetes\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.201473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.197989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b311cbf3-5339-4afc-ba92-986ba83cde95-host-slash\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0d059ee8-9a60-4470-9110-1bf7ea3e0359-host\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-socket-dir\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-var-lib-kubelet\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88c33e16-a136-4dae-a73f-73901b742b10-os-release\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-run\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198605    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-multus-socket-dir-parent\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198616    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-sysctl-conf\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198650    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-var-lib-cni-multus\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198671    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-systemd\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-sys-fs\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198728    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-host-run-k8s-cni-cncf-io\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-etc-kubernetes\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-lib-modules\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.202139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.198924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-cni-binary-copy\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.203260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.202854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-tmp\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.203260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.202881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-etc-tuned\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.203428 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.203266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/afafc254-8fa2-4876-9789-425e1c2a8523-agent-certs\") pod \"konnectivity-agent-c5g6r\" (UID: \"afafc254-8fa2-4876-9789-425e1c2a8523\") " pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.207997 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.207971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rvjx\" (UniqueName: \"kubernetes.io/projected/b311cbf3-5339-4afc-ba92-986ba83cde95-kube-api-access-6rvjx\") pod \"iptables-alerter-xj2t7\" (UID: \"b311cbf3-5339-4afc-ba92-986ba83cde95\") " pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.208324 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.208294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjd66\" (UniqueName: \"kubernetes.io/projected/88c33e16-a136-4dae-a73f-73901b742b10-kube-api-access-wjd66\") pod \"multus-additional-cni-plugins-kg4rd\" (UID: \"88c33e16-a136-4dae-a73f-73901b742b10\") " pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.212572 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.212546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2mzt\" (UniqueName: \"kubernetes.io/projected/f548050d-50ab-4a9d-99dd-f8a0a78a5e70-kube-api-access-b2mzt\") pod \"multus-drrlv\" (UID: \"f548050d-50ab-4a9d-99dd-f8a0a78a5e70\") " pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.212686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.212598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7w98\" (UniqueName: \"kubernetes.io/projected/071228ff-2484-40ba-84cd-49677845886d-kube-api-access-c7w98\") pod \"node-resolver-bbdlw\" (UID: \"071228ff-2484-40ba-84cd-49677845886d\") " pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.213101 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.213073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvzr7\" (UniqueName: \"kubernetes.io/projected/fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393-kube-api-access-mvzr7\") pod \"tuned-fmmvm\" (UID: \"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393\") " pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.213628 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.213603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t72gz\" (UniqueName: \"kubernetes.io/projected/db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31-kube-api-access-t72gz\") pod \"aws-ebs-csi-driver-node-9ckr4\" (UID: \"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.215463 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.215444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4vz\" (UniqueName: \"kubernetes.io/projected/0d059ee8-9a60-4470-9110-1bf7ea3e0359-kube-api-access-hj4vz\") pod \"node-ca-ds69v\" (UID: \"0d059ee8-9a60-4470-9110-1bf7ea3e0359\") " pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.298493 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-env-overrides\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-dbus\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-slash\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-etc-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-log-socket\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-bin\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.298686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-config\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298807    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-systemd-units\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-var-lib-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-netd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovn-node-metrics-cert\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6gr2\" (UniqueName: \"kubernetes.io/projected/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-kube-api-access-t6gr2\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz7pd\" (UniqueName: \"kubernetes.io/projected/88c8b447-df78-4fb5-9932-c7154765ee06-kube-api-access-vz7pd\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-ovn\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-var-lib-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-kubelet\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-node-log\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-systemd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-script-lib\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-netns\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-dbus\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-kubelet-config\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-slash\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-log-socket\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-bin\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-cni-netd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299447    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-config\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.298958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-env-overrides\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299562    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.299586    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:37.299654 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.299658    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:37.799636721 +0000 UTC m=+3.096218476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-ovn-kubernetes\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299763    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-run-netns\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-systemd\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.299940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-node-log\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-systemd-units\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300142 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-etc-openvswitch\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300334 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b8e300-2322-41e9-830d-0f0fe72835b8-kubelet-config\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.300334 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-run-ovn\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300334 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300267    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovnkube-script-lib\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.300431 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.300333    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:37.300431 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.300377    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:37.800363773 +0000 UTC m=+3.096945527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:37.300431 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.300423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-host-kubelet\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.301939 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.301911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-ovn-node-metrics-cert\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.312840 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.312373    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:37.312840 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.312417    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:37.312840 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.312433    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:37.312840 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.312484    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:14:37.812469251 +0000 UTC m=+3.109051023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:37.315959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.315930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6gr2\" (UniqueName: \"kubernetes.io/projected/4ed0745b-06d8-4e46-8b07-9dbc3bead91c-kube-api-access-t6gr2\") pod \"ovnkube-node-qhttr\" (UID: \"4ed0745b-06d8-4e46-8b07-9dbc3bead91c\") " pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.316096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.315930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz7pd\" (UniqueName: \"kubernetes.io/projected/88c8b447-df78-4fb5-9932-c7154765ee06-kube-api-access-vz7pd\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.366688 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.366645    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:37.391366 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.391335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bbdlw"
Jul 10 02:14:37.398294 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.398274    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xj2t7"
Jul 10 02:14:37.409240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.409218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4"
Jul 10 02:14:37.421998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.421969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ds69v"
Jul 10 02:14:37.432026 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.432003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-drrlv"
Jul 10 02:14:37.440053 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.440031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:37.448556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.448536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmmvm"
Jul 10 02:14:37.456460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.456439    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kg4rd"
Jul 10 02:14:37.463618 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.463597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:14:37.804215 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.804125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:37.804215 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.804164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:37.804432 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.804288    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:37.804432 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.804308    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:37.804432 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.804357    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:38.804339566 +0000 UTC m=+4.100921328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:37.804432 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.804374    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:38.804366808 +0000 UTC m=+4.100948563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:37.905384 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:37.905349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:37.905602 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.905584    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:37.905666 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.905615    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:37.905666 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.905630    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:37.905815 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:37.905710    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:14:38.90568785 +0000 UTC m=+4.202269607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:38.125653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:38.125563    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:09:37 +0000 UTC" deadline="2028-03-29 14:41:43.739997906 +0000 UTC"
Jul 10 02:14:38.125653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:38.125593    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15084h27m5.614408288s"
Jul 10 02:14:38.425990 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.425956    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071228ff_2484_40ba_84cd_49677845886d.slice/crio-0b71bc27cb5e78cf4cc486835e6fa8c8af8d084aad5e3b7de0b708035685098f WatchSource:0}: Error finding container 0b71bc27cb5e78cf4cc486835e6fa8c8af8d084aad5e3b7de0b708035685098f: Status 404 returned error can't find the container with id 0b71bc27cb5e78cf4cc486835e6fa8c8af8d084aad5e3b7de0b708035685098f
Jul 10 02:14:38.427633 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.427608    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb311cbf3_5339_4afc_ba92_986ba83cde95.slice/crio-7a3e29efefaf507825e85bc541d8c647e26ea4c2faf8ba7bef5073e35e3c1b94 WatchSource:0}: Error finding container 7a3e29efefaf507825e85bc541d8c647e26ea4c2faf8ba7bef5073e35e3c1b94: Status 404 returned error can't find the container with id 7a3e29efefaf507825e85bc541d8c647e26ea4c2faf8ba7bef5073e35e3c1b94
Jul 10 02:14:38.431708 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.431672    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0cc19d_eae2_4bfb_9f9d_4d8fe30db393.slice/crio-0b29bb84d39f3ef0d094feae8830582974189b30fe00e9705b95cd062cea2e34 WatchSource:0}: Error finding container 0b29bb84d39f3ef0d094feae8830582974189b30fe00e9705b95cd062cea2e34: Status 404 returned error can't find the container with id 0b29bb84d39f3ef0d094feae8830582974189b30fe00e9705b95cd062cea2e34
Jul 10 02:14:38.432567 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.432481    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d059ee8_9a60_4470_9110_1bf7ea3e0359.slice/crio-a26608a6597531ca647a491098176ae523c62e25443faa8aa7cefe84044c00db WatchSource:0}: Error finding container a26608a6597531ca647a491098176ae523c62e25443faa8aa7cefe84044c00db: Status 404 returned error can't find the container with id a26608a6597531ca647a491098176ae523c62e25443faa8aa7cefe84044c00db
Jul 10 02:14:38.433586 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.433558    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed0745b_06d8_4e46_8b07_9dbc3bead91c.slice/crio-c9f00b60e0a3320dd0d236a392be6e003d65d4c5bf6c6f505981d31ee20d33cd WatchSource:0}: Error finding container c9f00b60e0a3320dd0d236a392be6e003d65d4c5bf6c6f505981d31ee20d33cd: Status 404 returned error can't find the container with id c9f00b60e0a3320dd0d236a392be6e003d65d4c5bf6c6f505981d31ee20d33cd
Jul 10 02:14:38.434710 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.434591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf548050d_50ab_4a9d_99dd_f8a0a78a5e70.slice/crio-e784a4bffca2a33dffaa4609da305ee2299cd6675c4736f99d41f2c4c2e9f297 WatchSource:0}: Error finding container e784a4bffca2a33dffaa4609da305ee2299cd6675c4736f99d41f2c4c2e9f297: Status 404 returned error can't find the container with id e784a4bffca2a33dffaa4609da305ee2299cd6675c4736f99d41f2c4c2e9f297
Jul 10 02:14:38.436482 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.436458    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafafc254_8fa2_4876_9789_425e1c2a8523.slice/crio-2764d0730f985c45d9ffcdf4e8aeb266e340685d4d86429ae76cc44089c971b7 WatchSource:0}: Error finding container 2764d0730f985c45d9ffcdf4e8aeb266e340685d4d86429ae76cc44089c971b7: Status 404 returned error can't find the container with id 2764d0730f985c45d9ffcdf4e8aeb266e340685d4d86429ae76cc44089c971b7
Jul 10 02:14:38.439088 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:14:38.439061    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb598ee5_b5aa_4ca1_b9d0_4bdf18e4fe31.slice/crio-799ddc7efbc338f3b8f01a8f00f04398a0b72e1d4fdc574d57efd7273accd4c9 WatchSource:0}: Error finding container 799ddc7efbc338f3b8f01a8f00f04398a0b72e1d4fdc574d57efd7273accd4c9: Status 404 returned error can't find the container with id 799ddc7efbc338f3b8f01a8f00f04398a0b72e1d4fdc574d57efd7273accd4c9
Jul 10 02:14:38.822328 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:38.822123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:38.822465 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:38.822356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:38.822465 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.822273    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:38.822550 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.822468    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:40.822450015 +0000 UTC m=+6.119031774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:38.822550 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.822500    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:38.822628 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.822560    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:40.82254431 +0000 UTC m=+6.119126069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:38.923285 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:38.923186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:38.923430 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.923341    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:38.923430 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.923358    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:38.923430 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.923368    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:38.923430 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:38.923423    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:14:40.92340712 +0000 UTC m=+6.219988894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:39.126779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.126716    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:09:37 +0000 UTC" deadline="2028-01-09 10:05:22.927375011 +0000 UTC"
Jul 10 02:14:39.126779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.126778    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13159h50m43.80060229s"
Jul 10 02:14:39.239755 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.238885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:39.239755 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:39.239034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:39.239755 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.239509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:39.239755 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:39.239618    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:39.241901 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.240202    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:39.241901 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:39.240440    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:39.255339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.255299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerStarted","Data":"647350fb59929bb5a22dda069a9340cf392e00fb6f18221c148907a2f9589041"}
Jul 10 02:14:39.257592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.257565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4" event={"ID":"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31","Type":"ContainerStarted","Data":"799ddc7efbc338f3b8f01a8f00f04398a0b72e1d4fdc574d57efd7273accd4c9"}
Jul 10 02:14:39.259231 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.259205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5g6r" event={"ID":"afafc254-8fa2-4876-9789-425e1c2a8523","Type":"ContainerStarted","Data":"2764d0730f985c45d9ffcdf4e8aeb266e340685d4d86429ae76cc44089c971b7"}
Jul 10 02:14:39.261454 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.261428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"c9f00b60e0a3320dd0d236a392be6e003d65d4c5bf6c6f505981d31ee20d33cd"}
Jul 10 02:14:39.265358 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.265291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-drrlv" event={"ID":"f548050d-50ab-4a9d-99dd-f8a0a78a5e70","Type":"ContainerStarted","Data":"e784a4bffca2a33dffaa4609da305ee2299cd6675c4736f99d41f2c4c2e9f297"}
Jul 10 02:14:39.268125 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.268097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xj2t7" event={"ID":"b311cbf3-5339-4afc-ba92-986ba83cde95","Type":"ContainerStarted","Data":"7a3e29efefaf507825e85bc541d8c647e26ea4c2faf8ba7bef5073e35e3c1b94"}
Jul 10 02:14:39.287709 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.287683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bbdlw" event={"ID":"071228ff-2484-40ba-84cd-49677845886d","Type":"ContainerStarted","Data":"0b71bc27cb5e78cf4cc486835e6fa8c8af8d084aad5e3b7de0b708035685098f"}
Jul 10 02:14:39.299627 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.295836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" event={"ID":"16a2c0a00cdbe9d77d0bff3f91f1c3e7","Type":"ContainerStarted","Data":"342788a096b0e2d7e44437e038f203855e027309bd9d6a2c67f55d6f481cc778"}
Jul 10 02:14:39.299627 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.298834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ds69v" event={"ID":"0d059ee8-9a60-4470-9110-1bf7ea3e0359","Type":"ContainerStarted","Data":"a26608a6597531ca647a491098176ae523c62e25443faa8aa7cefe84044c00db"}
Jul 10 02:14:39.302822 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.302739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmmvm" event={"ID":"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393","Type":"ContainerStarted","Data":"0b29bb84d39f3ef0d094feae8830582974189b30fe00e9705b95cd062cea2e34"}
Jul 10 02:14:39.308167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.307197    2579 generic.go:356] "Generic (PLEG): container finished" podID="281d1b3a45b212ba9681d002edffe820" containerID="17d994aa7e485857e6b0bf2c9dcbf77ee058d34a7460b46a9b54044cdc805096" exitCode=0
Jul 10 02:14:39.308167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.307235    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerDied","Data":"17d994aa7e485857e6b0bf2c9dcbf77ee058d34a7460b46a9b54044cdc805096"}
Jul 10 02:14:39.333498 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:39.333416    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" podStartSLOduration=2.333395951 podStartE2EDuration="2.333395951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:39.311988412 +0000 UTC m=+4.608570191" watchObservedRunningTime="2026-07-10 02:14:39.333395951 +0000 UTC m=+4.629977936"
Jul 10 02:14:40.339363 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:40.338887    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerStarted","Data":"0913bd01f14952575b44673e232e9b8623b753f0646ec00d06fda89f6a279f8f"}
Jul 10 02:14:40.841285 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:40.841232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:40.841285 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:40.841286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:40.841533 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.841409    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:40.841533 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.841473    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:44.841452444 +0000 UTC m=+10.138034213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:40.841650 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.841550    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:40.841650 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.841589    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:44.841575254 +0000 UTC m=+10.138157023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:40.942769 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:40.942497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:40.942769 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.942724    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:40.942769 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.942747    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:40.942769 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.942760    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:40.943126 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:40.942839    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:14:44.94281866 +0000 UTC m=+10.239400426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:41.241740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:41.241912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:41.242320    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:41.242414    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:41.242476    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:41.242664 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:41.242549    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:43.241592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:43.241501    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:43.242076 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:43.241631    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:43.242076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:43.241710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:43.242076 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:43.241826    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:43.242076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:43.241878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:43.242076 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:43.241956    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:44.878122 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:44.878081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:44.878122 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:44.878128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:44.878631 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.878250    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:44.878631 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.878267    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:44.878631 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.878306    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.878290378 +0000 UTC m=+18.174872137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:44.878631 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.878343    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.878320816 +0000 UTC m=+18.174902576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:44.978748 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:44.978692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:44.978922 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.978876    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:44.978922 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.978894    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:44.978922 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.978907    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:44.979103 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:44.978972    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.978950771 +0000 UTC m=+18.275532551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:45.240235 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:45.240153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:45.240393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:45.240285    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:45.240393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:45.240319    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:45.240393 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:45.240346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:45.240536 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:45.240402    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:45.240536 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:45.240511    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:47.241038    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:47.241151    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:47.241445    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:47.241519    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:47.241553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:47.241780 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:47.241604    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:49.238804 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:49.238769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:49.239271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:49.238771    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:49.239271 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:49.238906    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:49.239271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:49.238950    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:49.239271 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:49.239058    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:49.239271 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:49.239141    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:51.239913 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:51.239877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:51.240364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:51.239982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:51.240364 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:51.239992    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:51.240364 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:51.240064    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:51.240364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:51.239877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:51.240364 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:51.240145    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:52.938539 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:52.938506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:52.938539 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:52.938548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:52.939092 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:52.938673    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:52.939092 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:52.938750    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:08.938716712 +0000 UTC m=+34.235298475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:52.939092 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:52.938771    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:52.939092 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:52.938821    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:08.938809642 +0000 UTC m=+34.235391398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:53.039100 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:53.039062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:53.039286 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.039267    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:53.039357 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.039290    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:53.039357 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.039302    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:53.039428 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.039359    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:15:09.039342146 +0000 UTC m=+34.335923921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:53.239086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:53.239007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:53.239250 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:53.239006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:53.239250 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:53.239138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:53.239250 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.239133    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:53.239414 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.239271    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:53.239414 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:53.239359    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:55.239693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:55.239661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:55.240190 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:55.239787    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:55.240190 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:55.239833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:55.240190 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:55.239913    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:55.240190 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:55.239936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:55.240190 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:55.240007    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:57.239536 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:57.239503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:57.240112 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:57.239511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:57.240112 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:57.239511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:57.240112 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:57.239639    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:57.240112 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:57.239742    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:57.240112 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:57.239883    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:58.379513 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.379202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bbdlw" event={"ID":"071228ff-2484-40ba-84cd-49677845886d","Type":"ContainerStarted","Data":"b5b5372811ad07654d841fde2cb33ea28b80089fc07d64eca316818822a79de8"}
Jul 10 02:14:58.380627 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.380608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ds69v" event={"ID":"0d059ee8-9a60-4470-9110-1bf7ea3e0359","Type":"ContainerStarted","Data":"cab1b21daef7d759fd9cc4aa51acb30faaec07c9b706c96cac4c3ee9dbbd4dbd"}
Jul 10 02:14:58.382001 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.381979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmmvm" event={"ID":"fd0cc19d-eae2-4bfb-9f9d-4d8fe30db393","Type":"ContainerStarted","Data":"f6c53ffc0b031ac50d06cb02e0bd0780908d4c72ffe0bc16caeb945c5e4fa83f"}
Jul 10 02:14:58.385952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.385393    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="2c73f9277a292ea2ed4f5cb69d31fb405aa3a337cb989db3624cf682ca84904f" exitCode=0
Jul 10 02:14:58.385952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.385693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"2c73f9277a292ea2ed4f5cb69d31fb405aa3a337cb989db3624cf682ca84904f"}
Jul 10 02:14:58.389124 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.389103    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4" event={"ID":"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31","Type":"ContainerStarted","Data":"f53292cb439a8604a47eab5bff5417ef39d092706b0cb0d9aa0aa515be6d0357"}
Jul 10 02:14:58.390457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.390429    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5g6r" event={"ID":"afafc254-8fa2-4876-9789-425e1c2a8523","Type":"ContainerStarted","Data":"a820861c093d46a9c7f069d03525d11bad1619469774739b9fcfdc8fb9b0cff3"}
Jul 10 02:14:58.392204 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392188    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:14:58.392505 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392487    2579 generic.go:356] "Generic (PLEG): container finished" podID="4ed0745b-06d8-4e46-8b07-9dbc3bead91c" containerID="c55b026f25e3a5096195031d934410d6bae8a58fe5e8230b357c4cd2e8da4152" exitCode=1
Jul 10 02:14:58.392561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"4cda79a5e4675f46565b8de348c40b9fe414fc32f6b51e6beb0df318700ae4b6"}
Jul 10 02:14:58.392617 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"77f29c86ead2d4edae4cea58532d433929a3656b6d5b55ea1b2d1e3d0539bfc1"}
Jul 10 02:14:58.392670 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392659    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerDied","Data":"c55b026f25e3a5096195031d934410d6bae8a58fe5e8230b357c4cd2e8da4152"}
Jul 10 02:14:58.392718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.392707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"b9a847e9f11b8d750474ab2e4b799fc42986948f5acbf893e5827f4e11932340"}
Jul 10 02:14:58.394337 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.394318    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-drrlv" event={"ID":"f548050d-50ab-4a9d-99dd-f8a0a78a5e70","Type":"ContainerStarted","Data":"ecb7163a516802372523f7cd96f8038914fcc2ec16a9903c04da08d70ef6ac93"}
Jul 10 02:14:58.398334 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.398290    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bbdlw" podStartSLOduration=4.039558289 podStartE2EDuration="23.398273266s" totalImagesPullingTime="19.358714977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.397822189 +0000 UTC m=+23.694403966" watchObservedRunningTime="2026-07-10 02:14:58.398273266 +0000 UTC m=+23.694855043"
Jul 10 02:14:58.398834 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.398800    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" podStartSLOduration=21.398791378 podStartE2EDuration="21.398791378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:40.357684398 +0000 UTC m=+5.654266176" watchObservedRunningTime="2026-07-10 02:14:58.398791378 +0000 UTC m=+23.695373156"
Jul 10 02:14:58.416718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.416687    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-drrlv" podStartSLOduration=3.010900948 podStartE2EDuration="22.41667562s" totalImagesPullingTime="19.405774672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.416351307 +0000 UTC m=+23.712933083" watchObservedRunningTime="2026-07-10 02:14:58.41667562 +0000 UTC m=+23.713257395"
Jul 10 02:14:58.432614 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.432573    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c5g6r" podStartSLOduration=7.49005848 podStartE2EDuration="22.432561438s" totalImagesPullingTime="14.942502958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.432455077 +0000 UTC m=+23.729036852" watchObservedRunningTime="2026-07-10 02:14:58.432561438 +0000 UTC m=+23.729143192"
Jul 10 02:14:58.495236 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.495198    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ds69v" podStartSLOduration=3.146764162 podStartE2EDuration="22.495185122s" totalImagesPullingTime="19.34842096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.476653748 +0000 UTC m=+23.773235523" watchObservedRunningTime="2026-07-10 02:14:58.495185122 +0000 UTC m=+23.791766895"
Jul 10 02:14:58.495410 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.495393    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fmmvm" podStartSLOduration=3.115084532 podStartE2EDuration="22.495386252s" totalImagesPullingTime="19.38030172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.495313378 +0000 UTC m=+23.791895136" watchObservedRunningTime="2026-07-10 02:14:58.495386252 +0000 UTC m=+23.791968026"
Jul 10 02:14:58.693568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.693476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:58.694197 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:58.694175    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:59.238507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.238477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:14:59.238718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.238473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:14:59.238718 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:59.238582    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:14:59.238718 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:59.238681    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:14:59.238718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.238473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:14:59.238936 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:14:59.238796    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:14:59.386698 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.386422    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 02:14:59.398409 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.398378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xj2t7" event={"ID":"b311cbf3-5339-4afc-ba92-986ba83cde95","Type":"ContainerStarted","Data":"a684488fb115462cba3fe642a0ac959ec31053ff86d20f9a3dd9cd8d9981c322"}
Jul 10 02:14:59.400359 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.400330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4" event={"ID":"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31","Type":"ContainerStarted","Data":"a0b101635a75e71ebf65be5f4060e1ad5bb66a7585d6616b91a65b03fcd5ef83"}
Jul 10 02:14:59.403507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.403486    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:14:59.403928 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.403907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"1029326ae3552721f230266a7a877a55152ebdea091795541e1c244f9e5b1ec4"}
Jul 10 02:14:59.404015 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.403986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"d033ace03092cee2904b489fa2599885f61c6b646a7a7eb80dcfdde261372481"}
Jul 10 02:14:59.404494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.404457    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:59.404585 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.404552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c5g6r"
Jul 10 02:14:59.417678 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:14:59.417633    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xj2t7" podStartSLOduration=5.060185324 podStartE2EDuration="24.417619983s" totalImagesPullingTime="19.357434659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:59.417040818 +0000 UTC m=+24.713622594" watchObservedRunningTime="2026-07-10 02:14:59.417619983 +0000 UTC m=+24.714201758"
Jul 10 02:15:00.153141 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:00.153038    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:14:59.386442567Z","UUID":"10a0d417-1559-4389-ba57-296f2b33f632","Handler":null,"Name":"","Endpoint":""}
Jul 10 02:15:00.154965 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:00.154934    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 02:15:00.155096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:00.154974    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 02:15:01.239609 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.239330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:01.240164 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.239330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:01.240164 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:01.239662    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:01.240164 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:01.239754    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:01.240164 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.239344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:01.240164 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:01.239843    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:01.411037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.411002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4" event={"ID":"db598ee5-b5aa-4ca1-b9d0-4bdf18e4fe31","Type":"ContainerStarted","Data":"bedb68bc2d46b792b5fc882b045ec2524e9967b4bfe51133292187af03eaae4d"}
Jul 10 02:15:01.414498 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.414480    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:15:01.414864 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.414849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"83ec214a1d1e7fdc32dcf663823ab4ae9f3b75f9cbe3764bbead32fe2165d19d"}
Jul 10 02:15:01.431601 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:01.431560    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ckr4" podStartSLOduration=3.700669272 podStartE2EDuration="25.431545932s" totalImagesPullingTime="21.73087666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:01.431512686 +0000 UTC m=+26.728094452" watchObservedRunningTime="2026-07-10 02:15:01.431545932 +0000 UTC m=+26.728127701"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:03.239527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:03.239876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:03.239912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:03.240016    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:03.240068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:03.242661 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:03.240210    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:05.239265 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.239114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:05.239847 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:05.239328    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:05.239847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.239244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:05.239847 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:05.239384    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:05.239847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.239204    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:05.239847 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:05.239429    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:05.425389 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.425351    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="d52c58df16e478a00b1d9efdaf15e341d285ea1105e5d29596b9032eb6d5aab0" exitCode=0
Jul 10 02:15:05.425571 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.425404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"d52c58df16e478a00b1d9efdaf15e341d285ea1105e5d29596b9032eb6d5aab0"}
Jul 10 02:15:05.429155 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.429138    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:15:05.429488 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.429466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"a5c6340781ac75b5e841278cdf408c61fae10f8e12974627bc1ede82ecf58228"}
Jul 10 02:15:05.429722 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.429703    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:05.429899 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.429885    2579 scope.go:117] "RemoveContainer" containerID="c55b026f25e3a5096195031d934410d6bae8a58fe5e8230b357c4cd2e8da4152"
Jul 10 02:15:05.446617 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:05.446600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:06.436428 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.436401    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:15:06.436928 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.436816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" event={"ID":"4ed0745b-06d8-4e46-8b07-9dbc3bead91c","Type":"ContainerStarted","Data":"49ca1a36f5c90f35ded2997c3733f997b373468faf06ddfb9c15453dea536ef8"}
Jul 10 02:15:06.437076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.437055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:06.437144 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.437129    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:06.454216 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.454195    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:06.500915 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:06.500862    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr" podStartSLOduration=11.058757039 podStartE2EDuration="30.500846341s" totalImagesPullingTime="19.442089302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:06.469056767 +0000 UTC m=+31.765638544" watchObservedRunningTime="2026-07-10 02:15:06.500846341 +0000 UTC m=+31.797428115"
Jul 10 02:15:07.238796 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:07.238710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:07.238969 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:07.238710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:07.238969 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:07.238846    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:07.238969 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:07.238841    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:07.238969 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:07.238927    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:07.239161 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:07.238998    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:07.441198 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:07.441169    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="9ad34e47a81530cb1f23172c0d6e79f174707a34e53a10cd0352df13f537cfda" exitCode=0
Jul 10 02:15:07.441533 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:07.441257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"9ad34e47a81530cb1f23172c0d6e79f174707a34e53a10cd0352df13f537cfda"}
Jul 10 02:15:08.445169 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:08.445138    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="5f0621974706073bb7530f07028c95f736bc89710c7463b6242a913d0b229f15" exitCode=0
Jul 10 02:15:08.445482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:08.445220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"5f0621974706073bb7530f07028c95f736bc89710c7463b6242a913d0b229f15"}
Jul 10 02:15:08.955993 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:08.955960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:08.956159 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:08.955999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:08.956159 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:08.956091    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:15:08.956159 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:08.956116    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:15:08.956159 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:08.956139    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret podName:38b8e300-2322-41e9-830d-0f0fe72835b8 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:40.956125719 +0000 UTC m=+66.252707475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret") pod "global-pull-secret-syncer-f5w9n" (UID: "38b8e300-2322-41e9-830d-0f0fe72835b8") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:15:08.956305 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:08.956166    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs podName:88c8b447-df78-4fb5-9932-c7154765ee06 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:40.95615171 +0000 UTC m=+66.252733463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs") pod "network-metrics-daemon-7mx5l" (UID: "88c8b447-df78-4fb5-9932-c7154765ee06") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:15:09.056753 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:09.056661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:09.056918 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.056848    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:15:09.056918 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.056873    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:15:09.056918 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.056886    2579 projected.go:196] Error preparing data for projected volume kube-api-access-lc8pc for pod openshift-network-diagnostics/network-check-target-pkhrv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:15:09.057079 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.056948    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc podName:13777cb7-d2b1-4ff2-8e57-36af56759f4d nodeName:}" failed. No retries permitted until 2026-07-10 02:15:41.056929475 +0000 UTC m=+66.353511231 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lc8pc" (UniqueName: "kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc") pod "network-check-target-pkhrv" (UID: "13777cb7-d2b1-4ff2-8e57-36af56759f4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:15:09.238651 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:09.238572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:09.238651 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:09.238574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:09.238864 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.238719    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:09.238864 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:09.238764    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:09.238864 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.238840    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:09.238977 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:09.238893    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:11.238912 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:11.238883    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:11.238912 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:11.238910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:11.239423 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:11.238881    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:11.239423 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:11.239017    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:11.239423 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:11.239101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:11.239423 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:11.239159    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:13.238574 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:13.238542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:13.239088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:13.238541    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:13.239088 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:13.238656    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:13.239088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:13.238683    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:13.239088 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:13.238777    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:13.239088 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:13.238873    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:14.463025 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.462797    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="906365ee65247d72a89de908bbb836366f6577849c568073e31b610ca4e46420" exitCode=0
Jul 10 02:15:14.463025 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.462854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"906365ee65247d72a89de908bbb836366f6577849c568073e31b610ca4e46420"}
Jul 10 02:15:14.571556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.571472    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mx5l"]
Jul 10 02:15:14.571845 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.571821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:14.572682 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:14.572358    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:14.576968 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.576940    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5w9n"]
Jul 10 02:15:14.577069 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.576994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pkhrv"]
Jul 10 02:15:14.577069 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.577047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:14.577183 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:14.577117    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:14.577298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:14.577281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:14.577382 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:14.577365    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:15.468354 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:15.468173    2579 generic.go:356] "Generic (PLEG): container finished" podID="88c33e16-a136-4dae-a73f-73901b742b10" containerID="4a327627010983c61616032cc70a1311a78f81d2337b0c4142c9725ab6078f95" exitCode=0
Jul 10 02:15:15.468354 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:15.468253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerDied","Data":"4a327627010983c61616032cc70a1311a78f81d2337b0c4142c9725ab6078f95"}
Jul 10 02:15:16.239147 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:16.239107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:16.239310 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:16.239218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:16.239310 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:16.239228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:16.239310 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:16.239228    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:16.239310 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:16.239298    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:16.239440 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:16.239379    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:16.473403 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:16.473367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" event={"ID":"88c33e16-a136-4dae-a73f-73901b742b10","Type":"ContainerStarted","Data":"f84a399429ada10b2500ad0f6cd7b02972ab717d93ee6c0e2a8ef973fbfb1d9f"}
Jul 10 02:15:16.498454 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:16.498367    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kg4rd" podStartSLOduration=7.743033454 podStartE2EDuration="40.498353756s" totalImagesPullingTime="32.755320302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:16.497931882 +0000 UTC m=+41.794513670" watchObservedRunningTime="2026-07-10 02:15:16.498353756 +0000 UTC m=+41.794935530"
Jul 10 02:15:18.238868 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:18.238833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:18.239359 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:18.238878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:18.239359 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:18.238941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:18.239359 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:18.238967    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5w9n" podUID="38b8e300-2322-41e9-830d-0f0fe72835b8"
Jul 10 02:15:18.239359 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:18.239067    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pkhrv" podUID="13777cb7-d2b1-4ff2-8e57-36af56759f4d"
Jul 10 02:15:18.239359 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:18.239134    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7mx5l" podUID="88c8b447-df78-4fb5-9932-c7154765ee06"
Jul 10 02:15:19.501980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.501953    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeReady"
Jul 10 02:15:19.502401 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.502089    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 02:15:19.552670 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.552589    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kcx59"]
Jul 10 02:15:19.571158 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.571129    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rbbw6"]
Jul 10 02:15:19.571285 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.571240    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.574170 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.574048    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 02:15:19.574170 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.574061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 02:15:19.574644 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.574483    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 02:15:19.574644 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.574490    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l6nlg\""
Jul 10 02:15:19.574644 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.574567    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 02:15:19.584984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.584965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kcx59"]
Jul 10 02:15:19.585064 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.585002    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbbw6"]
Jul 10 02:15:19.585101 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.585073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.587944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.587925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 02:15:19.588022 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.587925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2cc4f\""
Jul 10 02:15:19.588268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.588250    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 02:15:19.588357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.588320    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 02:15:19.638852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-cert\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.638852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-crio-socket\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.639020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-data-volume\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.639020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.639020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6xkn\" (UniqueName: \"kubernetes.io/projected/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-api-access-v6xkn\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.639020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.638967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxfbb\" (UniqueName: \"kubernetes.io/projected/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-kube-api-access-hxfbb\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.639152 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.639065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.655335 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.655309    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zsqwg"]
Jul 10 02:15:19.675315 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.675290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zsqwg"]
Jul 10 02:15:19.675436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.675413    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.678090 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.678070    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 02:15:19.678215 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.678162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 02:15:19.678296 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.678277    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zcw54\""
Jul 10 02:15:19.739499 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-tmp-dir\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.739636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-config-volume\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.739636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739585    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-cert\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.739636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-crio-socket\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r64bt\" (UniqueName: \"kubernetes.io/projected/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-kube-api-access-r64bt\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-data-volume\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-metrics-tls\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739785    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xkn\" (UniqueName: \"kubernetes.io/projected/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-api-access-v6xkn\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-crio-socket\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.739870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.739811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxfbb\" (UniqueName: \"kubernetes.io/projected/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-kube-api-access-hxfbb\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.740186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.740068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-data-volume\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.740349 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.740332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.743612 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.743591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.743612 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.743604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-cert\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.747855 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.747837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6xkn\" (UniqueName: \"kubernetes.io/projected/4adcdc5b-595f-4a8b-8315-fc2fb442f8ef-kube-api-access-v6xkn\") pod \"insights-runtime-extractor-kcx59\" (UID: \"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef\") " pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.747934 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.747886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxfbb\" (UniqueName: \"kubernetes.io/projected/2c747519-bdd7-4d10-b8ea-8ddccddee4dc-kube-api-access-hxfbb\") pod \"ingress-canary-rbbw6\" (UID: \"2c747519-bdd7-4d10-b8ea-8ddccddee4dc\") " pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.841077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.840988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-config-volume\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.841077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.841030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r64bt\" (UniqueName: \"kubernetes.io/projected/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-kube-api-access-r64bt\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.841077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.841053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-metrics-tls\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.841335 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.841107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-tmp-dir\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.841489 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.841466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-tmp-dir\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.841664 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.841642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-config-volume\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.843448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.843433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-metrics-tls\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.849160 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.849141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r64bt\" (UniqueName: \"kubernetes.io/projected/c6fa3dcb-01a3-47a5-8df7-65c08b4cced5-kube-api-access-r64bt\") pod \"dns-default-zsqwg\" (UID: \"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5\") " pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:19.881119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.881095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kcx59"
Jul 10 02:15:19.893251 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.893213    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbbw6"
Jul 10 02:15:19.983586 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:19.983546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:20.099952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.099744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kcx59"]
Jul 10 02:15:20.100926 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.100902    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbbw6"]
Jul 10 02:15:20.103333 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:20.103308    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adcdc5b_595f_4a8b_8315_fc2fb442f8ef.slice/crio-b2dac274b14ccf5a9e900894e45ee30f2267671c449833c2214872bdbd95a183 WatchSource:0}: Error finding container b2dac274b14ccf5a9e900894e45ee30f2267671c449833c2214872bdbd95a183: Status 404 returned error can't find the container with id b2dac274b14ccf5a9e900894e45ee30f2267671c449833c2214872bdbd95a183
Jul 10 02:15:20.105353 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:20.105324    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c747519_bdd7_4d10_b8ea_8ddccddee4dc.slice/crio-61d7319de352eecc1b033c2ec34f1bb2a515efe48ca161294eea19864b12aafb WatchSource:0}: Error finding container 61d7319de352eecc1b033c2ec34f1bb2a515efe48ca161294eea19864b12aafb: Status 404 returned error can't find the container with id 61d7319de352eecc1b033c2ec34f1bb2a515efe48ca161294eea19864b12aafb
Jul 10 02:15:20.146374 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.146347    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zsqwg"]
Jul 10 02:15:20.149122 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:20.149101    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fa3dcb_01a3_47a5_8df7_65c08b4cced5.slice/crio-6d1e6e15ce022db3162d05cb4f0c53363d31f837886a92bd54f80ba201b4f74f WatchSource:0}: Error finding container 6d1e6e15ce022db3162d05cb4f0c53363d31f837886a92bd54f80ba201b4f74f: Status 404 returned error can't find the container with id 6d1e6e15ce022db3162d05cb4f0c53363d31f837886a92bd54f80ba201b4f74f
Jul 10 02:15:20.238831 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.238801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:20.238831 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.238823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:20.238997 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.238835    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:20.242587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242123    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pmctl\""
Jul 10 02:15:20.242587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-htmn4\""
Jul 10 02:15:20.242587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242243    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:15:20.242587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242258    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:15:20.242587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242455    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:15:20.242944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.242906    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:15:20.268692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.268673    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"]
Jul 10 02:15:20.296821 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.296803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"]
Jul 10 02:15:20.296905 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.296882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:20.300531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.300503    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 10 02:15:20.300627 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.300531    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tmg4m\""
Jul 10 02:15:20.346123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.346063    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f3ff963-ff77-447a-8642-42edd4a71ef3-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pvphl\" (UID: \"2f3ff963-ff77-447a-8642-42edd4a71ef3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:20.447180 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.447148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f3ff963-ff77-447a-8642-42edd4a71ef3-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pvphl\" (UID: \"2f3ff963-ff77-447a-8642-42edd4a71ef3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:20.450492 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.450473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2f3ff963-ff77-447a-8642-42edd4a71ef3-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-pvphl\" (UID: \"2f3ff963-ff77-447a-8642-42edd4a71ef3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:20.483083 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.483053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcx59" event={"ID":"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef","Type":"ContainerStarted","Data":"b2dac274b14ccf5a9e900894e45ee30f2267671c449833c2214872bdbd95a183"}
Jul 10 02:15:20.484052 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.484034    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsqwg" event={"ID":"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5","Type":"ContainerStarted","Data":"6d1e6e15ce022db3162d05cb4f0c53363d31f837886a92bd54f80ba201b4f74f"}
Jul 10 02:15:20.485051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.485025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbbw6" event={"ID":"2c747519-bdd7-4d10-b8ea-8ddccddee4dc","Type":"ContainerStarted","Data":"61d7319de352eecc1b033c2ec34f1bb2a515efe48ca161294eea19864b12aafb"}
Jul 10 02:15:20.604722 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.604645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:20.773026 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:20.772995    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"]
Jul 10 02:15:20.776815 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:20.776776    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3ff963_ff77_447a_8642_42edd4a71ef3.slice/crio-8807c463f9a60a33e6b342836c34a3f0c118de02c2e81a18209c19b9f6610452 WatchSource:0}: Error finding container 8807c463f9a60a33e6b342836c34a3f0c118de02c2e81a18209c19b9f6610452: Status 404 returned error can't find the container with id 8807c463f9a60a33e6b342836c34a3f0c118de02c2e81a18209c19b9f6610452
Jul 10 02:15:21.491386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:21.491291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl" event={"ID":"2f3ff963-ff77-447a-8642-42edd4a71ef3","Type":"ContainerStarted","Data":"8807c463f9a60a33e6b342836c34a3f0c118de02c2e81a18209c19b9f6610452"}
Jul 10 02:15:27.507396 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.507159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcx59" event={"ID":"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef","Type":"ContainerStarted","Data":"bbbaca936218c80e74a42bbb49c8bfc8d1a0c4fbf84461246269967fd032a576"}
Jul 10 02:15:27.508806 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.508773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsqwg" event={"ID":"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5","Type":"ContainerStarted","Data":"8e922c1fa1afeff628cc0cc2bc1fafc3a74923aba430400bc9778b25af62c07b"}
Jul 10 02:15:27.508964 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.508828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zsqwg" event={"ID":"c6fa3dcb-01a3-47a5-8df7-65c08b4cced5","Type":"ContainerStarted","Data":"db4e01ea2e578d93d4a5c59e5494d897549aed6031176b74b9083b27adab5208"}
Jul 10 02:15:27.509024 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.508986    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:27.510256 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.510234    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl" event={"ID":"2f3ff963-ff77-447a-8642-42edd4a71ef3","Type":"ContainerStarted","Data":"16d22c73b9b25e967d7b142baa1b418c575b184d28c0768ba8f53d6cea3751a7"}
Jul 10 02:15:27.510379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.510360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:27.512073 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.512047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbbw6" event={"ID":"2c747519-bdd7-4d10-b8ea-8ddccddee4dc","Type":"ContainerStarted","Data":"560b7729f640fd45dfba934f230ca6db6ed0410f5299de1b4a1b0bdd939234bd"}
Jul 10 02:15:27.516379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.516356    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl"
Jul 10 02:15:27.532674 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.532633    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zsqwg" podStartSLOduration=1.843819221 podStartE2EDuration="8.53262146s" totalImagesPullingTime="6.688802239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:27.532014535 +0000 UTC m=+52.828596309" watchObservedRunningTime="2026-07-10 02:15:27.53262146 +0000 UTC m=+52.829203235"
Jul 10 02:15:27.564835 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.564797    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rbbw6" podStartSLOduration=1.8027071399999999 podStartE2EDuration="8.564782379s" totalImagesPullingTime="6.762075239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:27.547949261 +0000 UTC m=+52.844531040" watchObservedRunningTime="2026-07-10 02:15:27.564782379 +0000 UTC m=+52.861364154"
Jul 10 02:15:27.565058 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.565039    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-pvphl" podStartSLOduration=1.5044659280000001 podStartE2EDuration="7.565033234s" totalImagesPullingTime="6.060567306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:27.564577641 +0000 UTC m=+52.861159416" watchObservedRunningTime="2026-07-10 02:15:27.565033234 +0000 UTC m=+52.861615008"
Jul 10 02:15:27.632901 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.632867    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-fbwrd"]
Jul 10 02:15:27.655829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.655800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fbwrd"]
Jul 10 02:15:27.655952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.655894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:27.658559 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.658540    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 02:15:27.658899 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.658878    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 02:15:27.659050 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.659030    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qtxbb\""
Jul 10 02:15:27.706291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.706264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlrsj\" (UniqueName: \"kubernetes.io/projected/1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb-kube-api-access-rlrsj\") pod \"downloads-68c9cbb497-fbwrd\" (UID: \"1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb\") " pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:27.807301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.807272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlrsj\" (UniqueName: \"kubernetes.io/projected/1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb-kube-api-access-rlrsj\") pod \"downloads-68c9cbb497-fbwrd\" (UID: \"1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb\") " pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:27.818603 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.818577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlrsj\" (UniqueName: \"kubernetes.io/projected/1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb-kube-api-access-rlrsj\") pod \"downloads-68c9cbb497-fbwrd\" (UID: \"1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb\") " pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:27.964599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:27.964565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:28.230397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.230180    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fbwrd"]
Jul 10 02:15:28.234224 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:28.234168    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1bc1f5_da85_4c86_bc33_6f6cef5ee6fb.slice/crio-a8505611a3b2901631f0545d9157bc872f5721bcb34419759f1eb13f9be6fb21 WatchSource:0}: Error finding container a8505611a3b2901631f0545d9157bc872f5721bcb34419759f1eb13f9be6fb21: Status 404 returned error can't find the container with id a8505611a3b2901631f0545d9157bc872f5721bcb34419759f1eb13f9be6fb21
Jul 10 02:15:28.355188 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.355151    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rrpkh"]
Jul 10 02:15:28.367774 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.367701    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.371504 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.371478    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 02:15:28.371653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.371574    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 02:15:28.371887 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.371869    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 02:15:28.371965 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.371894    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 02:15:28.372132 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.372116    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 02:15:28.372813 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.372796    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-gp979\""
Jul 10 02:15:28.377069 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.377051    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rrpkh"]
Jul 10 02:15:28.411132 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.411101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.411271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.411138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.411271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.411164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1f85e19-00b6-4576-bd54-afca38e59c15-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.411271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.411216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttvx9\" (UniqueName: \"kubernetes.io/projected/c1f85e19-00b6-4576-bd54-afca38e59c15-kube-api-access-ttvx9\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.512642 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.512603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.512995 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.512650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.512995 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.512680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1f85e19-00b6-4576-bd54-afca38e59c15-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.512995 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.512701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttvx9\" (UniqueName: \"kubernetes.io/projected/c1f85e19-00b6-4576-bd54-afca38e59c15-kube-api-access-ttvx9\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.512995 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:28.512764    2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 10 02:15:28.512995 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:28.512828    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls podName:c1f85e19-00b6-4576-bd54-afca38e59c15 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:29.012809165 +0000 UTC m=+54.309390944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rrpkh" (UID: "c1f85e19-00b6-4576-bd54-afca38e59c15") : secret "prometheus-operator-tls" not found
Jul 10 02:15:28.513400 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.513377    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1f85e19-00b6-4576-bd54-afca38e59c15-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.515712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.515691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:28.517609 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.517587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fbwrd" event={"ID":"1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb","Type":"ContainerStarted","Data":"a8505611a3b2901631f0545d9157bc872f5721bcb34419759f1eb13f9be6fb21"}
Jul 10 02:15:28.527998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:28.527973    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttvx9\" (UniqueName: \"kubernetes.io/projected/c1f85e19-00b6-4576-bd54-afca38e59c15-kube-api-access-ttvx9\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:29.016540 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.016507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:29.019021 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.019001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1f85e19-00b6-4576-bd54-afca38e59c15-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rrpkh\" (UID: \"c1f85e19-00b6-4576-bd54-afca38e59c15\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:29.277519 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.277443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh"
Jul 10 02:15:29.428925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.428889    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rrpkh"]
Jul 10 02:15:29.523095 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.523053    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh" event={"ID":"c1f85e19-00b6-4576-bd54-afca38e59c15","Type":"ContainerStarted","Data":"f74e518c8f7dc445a9177ca7c7af835b1b4132f85fe2da674132fa08a7ba9aa3"}
Jul 10 02:15:29.525573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:29.525545    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcx59" event={"ID":"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef","Type":"ContainerStarted","Data":"adf2d71663df2735344be614a1e4f11a2286d863c3d4e76a2eed83d3dcbe265b"}
Jul 10 02:15:32.539173 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:32.538923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh" event={"ID":"c1f85e19-00b6-4576-bd54-afca38e59c15","Type":"ContainerStarted","Data":"a8ef203f4f4998bd1afe3dac8d299866fd4ff35655caaa12a3744934a4d88b15"}
Jul 10 02:15:32.539640 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:32.539213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh" event={"ID":"c1f85e19-00b6-4576-bd54-afca38e59c15","Type":"ContainerStarted","Data":"2639d8a04d80aeb269417cc7b03d7a1ac17f045fdc8a53bcf36284be5c374ff1"}
Jul 10 02:15:32.541824 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:32.541780    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kcx59" event={"ID":"4adcdc5b-595f-4a8b-8315-fc2fb442f8ef","Type":"ContainerStarted","Data":"2718c823a7f58298981a651822d543316891df3dfb0231baa9a840f891fcfcc1"}
Jul 10 02:15:32.559980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:32.559932    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rrpkh" podStartSLOduration=2.197204224 podStartE2EDuration="4.559916812s" totalImagesPullingTime="2.362712588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:32.558986791 +0000 UTC m=+57.855568569" watchObservedRunningTime="2026-07-10 02:15:32.559916812 +0000 UTC m=+57.856498594"
Jul 10 02:15:32.580058 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:32.580013    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kcx59" podStartSLOduration=2.140719681 podStartE2EDuration="13.579997557s" totalImagesPullingTime="11.439277876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:15:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:32.579305177 +0000 UTC m=+57.875886951" watchObservedRunningTime="2026-07-10 02:15:32.579997557 +0000 UTC m=+57.876579310"
Jul 10 02:15:34.817075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.817019    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"]
Jul 10 02:15:34.821025 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.821001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.823597 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.823574    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 10 02:15:34.823813 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.823787    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 10 02:15:34.823959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.823852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6mf8s\""
Jul 10 02:15:34.833865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.833844    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"]
Jul 10 02:15:34.836563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.836545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"]
Jul 10 02:15:34.836674 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.836640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.841469 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.841449    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 10 02:15:34.841574 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.841500    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 10 02:15:34.841884 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.841864    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lftn2\""
Jul 10 02:15:34.841998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.841892    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 10 02:15:34.851981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.851954    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8sb8m"]
Jul 10 02:15:34.854521 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.854500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"]
Jul 10 02:15:34.854657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.854638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.858047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.858028    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 02:15:34.858243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.858054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t929n\""
Jul 10 02:15:34.858243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.858055    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 02:15:34.858396 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.858055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 02:15:34.866376 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.866353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.866466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.866404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.866466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.866431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmcm9\" (UniqueName: \"kubernetes.io/projected/654697c0-2dad-4b00-bb43-7d2a826e76be-kube-api-access-bmcm9\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.866568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.866549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654697c0-2dad-4b00-bb43-7d2a826e76be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.967870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.967842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-sys\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968009 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.967885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.968009 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.967913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmcm9\" (UniqueName: \"kubernetes.io/projected/654697c0-2dad-4b00-bb43-7d2a826e76be-kube-api-access-bmcm9\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.968009 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.967953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7krqk\" (UniqueName: \"kubernetes.io/projected/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-api-access-7krqk\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968183 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968223 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.968223 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-metrics-client-ca\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968294 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61d636a1-904c-4677-acc9-4dd9aa55aa40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968294 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:34.968259    2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 10 02:15:34.968294 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968433 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:34.968318    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls podName:654697c0-2dad-4b00-bb43-7d2a826e76be nodeName:}" failed. No retries permitted until 2026-07-10 02:15:35.468298332 +0000 UTC m=+60.764880085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-9vmqp" (UID: "654697c0-2dad-4b00-bb43-7d2a826e76be") : secret "openshift-state-metrics-tls" not found
Jul 10 02:15:34.968433 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968524 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-root\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968524 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968524 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-wtmp\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968524 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:34.968667 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwn6g\" (UniqueName: \"kubernetes.io/projected/d25ba6ad-0cd9-40da-be81-51109673f2bb-kube-api-access-cwn6g\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968667 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-textfile\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:34.968667 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.968566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654697c0-2dad-4b00-bb43-7d2a826e76be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.969141 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.969125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654697c0-2dad-4b00-bb43-7d2a826e76be-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.971591 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.971568    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:34.984958 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:34.984937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmcm9\" (UniqueName: \"kubernetes.io/projected/654697c0-2dad-4b00-bb43-7d2a826e76be-kube-api-access-bmcm9\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:35.069078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.068982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-sys\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7krqk\" (UniqueName: \"kubernetes.io/projected/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-api-access-7krqk\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-sys\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-metrics-client-ca\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61d636a1-904c-4677-acc9-4dd9aa55aa40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-root\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-wtmp\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwn6g\" (UniqueName: \"kubernetes.io/projected/d25ba6ad-0cd9-40da-be81-51109673f2bb-kube-api-access-cwn6g\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-textfile\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-root\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-wtmp\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/61d636a1-904c-4677-acc9-4dd9aa55aa40-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.069863 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.069821    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-textfile\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.070182 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.070068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.070182 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.070102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-metrics-client-ca\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.083838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.083812    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 02:15:35.084397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.084383    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 02:15:35.084561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.084542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 10 02:15:35.084658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.084641    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 02:15:35.084899 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.084722    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 10 02:15:35.084899 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.084796    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 10 02:15:35.087655 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.087637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7krqk\" (UniqueName: \"kubernetes.io/projected/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-api-access-7krqk\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.090251 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:35.090151    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 02:15:35.090251 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:35.090221    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls podName:d25ba6ad-0cd9-40da-be81-51109673f2bb nodeName:}" failed. No retries permitted until 2026-07-10 02:15:35.590202793 +0000 UTC m=+60.886784555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls") pod "node-exporter-8sb8m" (UID: "d25ba6ad-0cd9-40da-be81-51109673f2bb") : secret "node-exporter-tls" not found
Jul 10 02:15:35.090967 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.090890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.091499 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.091201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.093457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.093429    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwn6g\" (UniqueName: \"kubernetes.io/projected/d25ba6ad-0cd9-40da-be81-51109673f2bb-kube-api-access-cwn6g\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.095404 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.095374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.096470 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.096446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.096861 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.096835    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d636a1-904c-4677-acc9-4dd9aa55aa40-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-4kx7x\" (UID: \"61d636a1-904c-4677-acc9-4dd9aa55aa40\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.150888 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.150826    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lftn2\""
Jul 10 02:15:35.157917 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.157889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"
Jul 10 02:15:35.306961 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.306921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-4kx7x"]
Jul 10 02:15:35.309644 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:35.309619    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d636a1_904c_4677_acc9_4dd9aa55aa40.slice/crio-839ca030fe6a41ff1d57f4597224a56fdfab649b0fa025705fde38a8366bf65a WatchSource:0}: Error finding container 839ca030fe6a41ff1d57f4597224a56fdfab649b0fa025705fde38a8366bf65a: Status 404 returned error can't find the container with id 839ca030fe6a41ff1d57f4597224a56fdfab649b0fa025705fde38a8366bf65a
Jul 10 02:15:35.472928 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.472848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:35.475583 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.475556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/654697c0-2dad-4b00-bb43-7d2a826e76be-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9vmqp\" (UID: \"654697c0-2dad-4b00-bb43-7d2a826e76be\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:35.555076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.555032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x" event={"ID":"61d636a1-904c-4677-acc9-4dd9aa55aa40","Type":"ContainerStarted","Data":"839ca030fe6a41ff1d57f4597224a56fdfab649b0fa025705fde38a8366bf65a"}
Jul 10 02:15:35.674854 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.674822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.677960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.677931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d25ba6ad-0cd9-40da-be81-51109673f2bb-node-exporter-tls\") pod \"node-exporter-8sb8m\" (UID: \"d25ba6ad-0cd9-40da-be81-51109673f2bb\") " pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.735314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.735235    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6mf8s\""
Jul 10 02:15:35.742593 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.742563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"
Jul 10 02:15:35.767480 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.767460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t929n\""
Jul 10 02:15:35.775669 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.775649    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8sb8m"
Jul 10 02:15:35.795843 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:35.795807    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25ba6ad_0cd9_40da_be81_51109673f2bb.slice/crio-649a2ac1d8e9622b3ec07884fb19aaa22f9d7e9254774beec60de75680eb5e71 WatchSource:0}: Error finding container 649a2ac1d8e9622b3ec07884fb19aaa22f9d7e9254774beec60de75680eb5e71: Status 404 returned error can't find the container with id 649a2ac1d8e9622b3ec07884fb19aaa22f9d7e9254774beec60de75680eb5e71
Jul 10 02:15:35.819795 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.818621    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:15:35.826807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.826753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.830088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.830018    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mns7f\""
Jul 10 02:15:35.830243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.830216    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 02:15:35.830596 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.830431    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 02:15:35.831169 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 02:15:35.831169 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 02:15:35.832483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831518    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 02:15:35.832483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831776    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 02:15:35.832483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831890    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 02:15:35.832483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.831969    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 02:15:35.832796 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.832558    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 02:15:35.841436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.841415    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:15:35.876764 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.876764 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876772    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876854    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5fw9\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.876932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.877024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.877069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.877109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.877460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.877179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.913531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.913412    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp"]
Jul 10 02:15:35.916992 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:35.916968    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654697c0_2dad_4b00_bb43_7d2a826e76be.slice/crio-2a394a9ba43e53a33a42880665881965282558f6560eb72307a10964db3a8659 WatchSource:0}: Error finding container 2a394a9ba43e53a33a42880665881965282558f6560eb72307a10964db3a8659: Status 404 returned error can't find the container with id 2a394a9ba43e53a33a42880665881965282558f6560eb72307a10964db3a8659
Jul 10 02:15:35.977800 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.977932 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.977932 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.977995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5fw9\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978290 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.978205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.978580 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:35.978383    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle podName:749eb7ac-4ddb-4da6-90ff-de53fee41af3 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:36.478360735 +0000 UTC m=+61.774942697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3") : configmap references non-existent config key: ca-bundle.crt
Jul 10 02:15:35.979498 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.979477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.980165 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:35.979656    2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 10 02:15:35.980165 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:35.979710    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls podName:749eb7ac-4ddb-4da6-90ff-de53fee41af3 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:36.479694501 +0000 UTC m=+61.776276274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3") : secret "alertmanager-main-tls" not found
Jul 10 02:15:35.980993 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.980918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.984491 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.984467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.985176 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.984809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.985263 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.985230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.985405 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.985358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.985609 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.985588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.986136 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.986092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.987262 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.986952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.987262 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.987219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:35.992202 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:35.992146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5fw9\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.481533 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.481500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.481694 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.481556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.482544 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.482522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.484462 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.484435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.567523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.567468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sb8m" event={"ID":"d25ba6ad-0cd9-40da-be81-51109673f2bb","Type":"ContainerStarted","Data":"649a2ac1d8e9622b3ec07884fb19aaa22f9d7e9254774beec60de75680eb5e71"}
Jul 10 02:15:36.571172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.571135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp" event={"ID":"654697c0-2dad-4b00-bb43-7d2a826e76be","Type":"ContainerStarted","Data":"fc6b6ea12dfe73c095846955e973b5456023a4b2784f2ca022c102010c6b7197"}
Jul 10 02:15:36.571413 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.571393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp" event={"ID":"654697c0-2dad-4b00-bb43-7d2a826e76be","Type":"ContainerStarted","Data":"b4aa23cbfec4842356206e50e0fbed893fed679ed85263070cbd0b3b4b026ca8"}
Jul 10 02:15:36.571547 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.571530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp" event={"ID":"654697c0-2dad-4b00-bb43-7d2a826e76be","Type":"ContainerStarted","Data":"2a394a9ba43e53a33a42880665881965282558f6560eb72307a10964db3a8659"}
Jul 10 02:15:36.739131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.739047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:15:36.764495 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.764470    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:15:36.782468 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.782444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.792807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.792777    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 02:15:36.792807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.792796    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 02:15:36.792960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.792835    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 02:15:36.792960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.792854    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-csw28\""
Jul 10 02:15:36.792960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.792796    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 02:15:36.793447 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.793272    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 02:15:36.803208 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.803186    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:15:36.884179 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.884622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.884622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884277    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.884622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.884622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.884622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.884447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dslkw\" (UniqueName: \"kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985336 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985511 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dslkw\" (UniqueName: \"kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985862 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.985862 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.985814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.986171 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.986145    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.986296 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.986275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.986643 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.986624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.988700 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.988681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.988870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.988840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:36.994849 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:36.994777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dslkw\" (UniqueName: \"kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw\") pod \"console-75ccb89555-6j52s\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") " pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:37.095572 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.095387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:37.522149 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.522066    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zsqwg"
Jul 10 02:15:37.583454 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.583299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sb8m" event={"ID":"d25ba6ad-0cd9-40da-be81-51109673f2bb","Type":"ContainerStarted","Data":"0bc61ae89eb08ba5b1068a7f37200068dbc89256ffb8d15e54a75576a324ed3f"}
Jul 10 02:15:37.588566 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.588287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x" event={"ID":"61d636a1-904c-4677-acc9-4dd9aa55aa40","Type":"ContainerStarted","Data":"47f1c2fea7f92b7d9abe1172d455ff16d1aceda1cfffa956880691be0707489d"}
Jul 10 02:15:37.605500 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.605429    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:15:37.653394 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.653244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:15:37.834651 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.834612    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65bfd8447f-g69pl"]
Jul 10 02:15:37.838588 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.838566    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.841888 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.841853    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dvu1aefht7itf\""
Jul 10 02:15:37.842019 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.841928    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 02:15:37.842174 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.842155    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 02:15:37.842240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.842229    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 02:15:37.842396 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.842378    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 02:15:37.842537 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.842158    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 02:15:37.842763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.842719    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rc69r\""
Jul 10 02:15:37.853809 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.853778    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65bfd8447f-g69pl"]
Jul 10 02:15:37.893867 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.893836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.893957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894041    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-grpc-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521a4d1c-c832-4ecb-a552-4ea144f712d0-metrics-client-ca\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894291 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.894597 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.894304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sg4n\" (UniqueName: \"kubernetes.io/projected/521a4d1c-c832-4ecb-a552-4ea144f712d0-kube-api-access-9sg4n\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-grpc-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521a4d1c-c832-4ecb-a552-4ea144f712d0-metrics-client-ca\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.997466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.996993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sg4n\" (UniqueName: \"kubernetes.io/projected/521a4d1c-c832-4ecb-a552-4ea144f712d0-kube-api-access-9sg4n\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:37.998906 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:37.998699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.001435 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.001295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/521a4d1c-c832-4ecb-a552-4ea144f712d0-metrics-client-ca\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.008568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.008540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.011557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.011479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.012488 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.012455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sg4n\" (UniqueName: \"kubernetes.io/projected/521a4d1c-c832-4ecb-a552-4ea144f712d0-kube-api-access-9sg4n\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.012648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.012489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.012897 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.012873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.014275 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.014227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-grpc-tls\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.015000 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.014956    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/521a4d1c-c832-4ecb-a552-4ea144f712d0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65bfd8447f-g69pl\" (UID: \"521a4d1c-c832-4ecb-a552-4ea144f712d0\") " pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.151116 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.151067    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:15:38.381478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.381338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65bfd8447f-g69pl"]
Jul 10 02:15:38.385159 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:38.385135    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521a4d1c_c832_4ecb_a552_4ea144f712d0.slice/crio-6b1bed6ec7a14a9e10b85b9650678c0dd4aafa2fc0ef8a55641aa2405c3bc833 WatchSource:0}: Error finding container 6b1bed6ec7a14a9e10b85b9650678c0dd4aafa2fc0ef8a55641aa2405c3bc833: Status 404 returned error can't find the container with id 6b1bed6ec7a14a9e10b85b9650678c0dd4aafa2fc0ef8a55641aa2405c3bc833
Jul 10 02:15:38.459968 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.459930    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qhttr"
Jul 10 02:15:38.597473 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.597438    2579 generic.go:356] "Generic (PLEG): container finished" podID="d25ba6ad-0cd9-40da-be81-51109673f2bb" containerID="0bc61ae89eb08ba5b1068a7f37200068dbc89256ffb8d15e54a75576a324ed3f" exitCode=0
Jul 10 02:15:38.597668 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.597538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sb8m" event={"ID":"d25ba6ad-0cd9-40da-be81-51109673f2bb","Type":"ContainerDied","Data":"0bc61ae89eb08ba5b1068a7f37200068dbc89256ffb8d15e54a75576a324ed3f"}
Jul 10 02:15:38.601313 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.600773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp" event={"ID":"654697c0-2dad-4b00-bb43-7d2a826e76be","Type":"ContainerStarted","Data":"b18f71e73922043c3db40e86ea8ce530cb98ed60fca56372ffea012337b2a26b"}
Jul 10 02:15:38.606010 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.605982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x" event={"ID":"61d636a1-904c-4677-acc9-4dd9aa55aa40","Type":"ContainerStarted","Data":"282b4a4494bacd5d3752135705125642c5f2a23bce4d04823551b697868a0d5b"}
Jul 10 02:15:38.606122 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.606041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x" event={"ID":"61d636a1-904c-4677-acc9-4dd9aa55aa40","Type":"ContainerStarted","Data":"62d001023a39888e29d631da7358db53bb4fb5cb021b4504dbebaaa7d60cf82f"}
Jul 10 02:15:38.610862 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.610674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccb89555-6j52s" event={"ID":"b43ede92-02c3-4e9e-b2f6-af06cce7820e","Type":"ContainerStarted","Data":"af1774fcfdcbabe959223207855144a71d01225c1c99e4ec0353f821d7860a35"}
Jul 10 02:15:38.612448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.612377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"5e63c432f84630283272f62ff884f64b4fa7e7509ac4644adf1abd59f85ff3c9"}
Jul 10 02:15:38.613764 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.613656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"6b1bed6ec7a14a9e10b85b9650678c0dd4aafa2fc0ef8a55641aa2405c3bc833"}
Jul 10 02:15:38.637243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.635988    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9vmqp" podStartSLOduration=2.467698083 podStartE2EDuration="4.635963378s" totalImagesPullingTime="2.168265295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:38.635718356 +0000 UTC m=+63.932300133" watchObservedRunningTime="2026-07-10 02:15:38.635963378 +0000 UTC m=+63.932545154"
Jul 10 02:15:38.654254 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:38.653840    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-4kx7x" podStartSLOduration=2.5183671309999998 podStartE2EDuration="4.653819973s" totalImagesPullingTime="2.135452842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:38.652983438 +0000 UTC m=+63.949565214" watchObservedRunningTime="2026-07-10 02:15:38.653819973 +0000 UTC m=+63.950401748"
Jul 10 02:15:39.230413 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.230384    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7dff7d7994-dkkl7"]
Jul 10 02:15:39.236521 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.236497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.241209 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.241186    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 02:15:39.241607 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.241585    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 10 02:15:39.241701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.241647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5ljp3tdkcnkac\""
Jul 10 02:15:39.241794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.241754    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x652n\""
Jul 10 02:15:39.241845 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.241802    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 10 02:15:39.242333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.242316    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 10 02:15:39.246541 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.246516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dff7d7994-dkkl7"]
Jul 10 02:15:39.313077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-metrics-server-audit-profiles\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01c18038-ca2e-40c2-85ab-40cc7caa12e8-audit-log\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-client-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8dwh\" (UniqueName: \"kubernetes.io/projected/01c18038-ca2e-40c2-85ab-40cc7caa12e8-kube-api-access-c8dwh\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-client-certs\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313221    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-tls\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.313417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.313246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413770 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-tls\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413770 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-metrics-server-audit-profiles\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413891    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01c18038-ca2e-40c2-85ab-40cc7caa12e8-audit-log\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-client-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8dwh\" (UniqueName: \"kubernetes.io/projected/01c18038-ca2e-40c2-85ab-40cc7caa12e8-kube-api-access-c8dwh\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.413980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.413976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-client-certs\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.414640 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.414609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01c18038-ca2e-40c2-85ab-40cc7caa12e8-audit-log\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.414768 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.414722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.415526 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.415479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01c18038-ca2e-40c2-85ab-40cc7caa12e8-metrics-server-audit-profiles\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.416797 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.416769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-tls\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.417010 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.416990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-secret-metrics-server-client-certs\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.417122 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.417102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01c18038-ca2e-40c2-85ab-40cc7caa12e8-client-ca-bundle\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.423933 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.423909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8dwh\" (UniqueName: \"kubernetes.io/projected/01c18038-ca2e-40c2-85ab-40cc7caa12e8-kube-api-access-c8dwh\") pod \"metrics-server-7dff7d7994-dkkl7\" (UID: \"01c18038-ca2e-40c2-85ab-40cc7caa12e8\") " pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.494158 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.493156    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"]
Jul 10 02:15:39.497996 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.497382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.500213 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.500192    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 10 02:15:39.500668 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.500495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w4bdt\""
Jul 10 02:15:39.505994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.505968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"]
Jul 10 02:15:39.583187 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.583151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:39.615560 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.615528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.615748 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.615571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k2sw\" (UniqueName: \"kubernetes.io/projected/5e1dfb10-c355-4395-978b-c895bab566ed-kube-api-access-6k2sw\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.633744 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.633491    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598" exitCode=0
Jul 10 02:15:39.633744 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.633607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598"}
Jul 10 02:15:39.639883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.638163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sb8m" event={"ID":"d25ba6ad-0cd9-40da-be81-51109673f2bb","Type":"ContainerStarted","Data":"5affbf380ee21a2d34b36b302a32e19b7f90120f32f54aebf7fd3937579b436a"}
Jul 10 02:15:39.639883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.638201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8sb8m" event={"ID":"d25ba6ad-0cd9-40da-be81-51109673f2bb","Type":"ContainerStarted","Data":"74bac96e9f24d54d2a4283b5bfa170393ecb5494c2469d2abf13b36deefa90d2"}
Jul 10 02:15:39.681623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.680970    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8sb8m" podStartSLOduration=3.890361126 podStartE2EDuration="5.680949137s" totalImagesPullingTime="1.651995738s" totalInitContainerRuntime="138.592273ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:39.680713132 +0000 UTC m=+64.977294911" watchObservedRunningTime="2026-07-10 02:15:39.680949137 +0000 UTC m=+64.977530913"
Jul 10 02:15:39.719058 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.717665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.719058 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.717892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k2sw\" (UniqueName: \"kubernetes.io/projected/5e1dfb10-c355-4395-978b-c895bab566ed-kube-api-access-6k2sw\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.719290 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:39.719209    2579 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 10 02:15:39.719290 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:39.719269    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert podName:5e1dfb10-c355-4395-978b-c895bab566ed nodeName:}" failed. No retries permitted until 2026-07-10 02:15:40.219248008 +0000 UTC m=+65.515829761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-z8vs7" (UID: "5e1dfb10-c355-4395-978b-c895bab566ed") : secret "monitoring-plugin-cert" not found
Jul 10 02:15:39.729251 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.729221    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k2sw\" (UniqueName: \"kubernetes.io/projected/5e1dfb10-c355-4395-978b-c895bab566ed-kube-api-access-6k2sw\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:39.752778 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.752751    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dff7d7994-dkkl7"]
Jul 10 02:15:39.756485 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:39.756457    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c18038_ca2e_40c2_85ab_40cc7caa12e8.slice/crio-032c562505f95756327f7c41fe2b7255680c09c72d6da37f8c663691248de64c WatchSource:0}: Error finding container 032c562505f95756327f7c41fe2b7255680c09c72d6da37f8c663691248de64c: Status 404 returned error can't find the container with id 032c562505f95756327f7c41fe2b7255680c09c72d6da37f8c663691248de64c
Jul 10 02:15:39.976574 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.975744    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"]
Jul 10 02:15:39.979804 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.979781    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.982678    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.982955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.983146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.983289    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.983919    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 10 02:15:39.986692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.984386    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vgnxm\""
Jul 10 02:15:39.991794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.991114    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"]
Jul 10 02:15:39.992358 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:39.992136    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 10 02:15:40.124074 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.123920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-metrics-client-ca\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124074 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.123978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124074 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-federate-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-serving-certs-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s86f\" (UniqueName: \"kubernetes.io/projected/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-kube-api-access-5s86f\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.124386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.124296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-federate-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-serving-certs-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s86f\" (UniqueName: \"kubernetes.io/projected/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-kube-api-access-5s86f\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-metrics-client-ca\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.225405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.226902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-serving-certs-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.227068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-metrics-client-ca\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.227819 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.227603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.228704 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.228657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.229333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.229306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.229718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.229688    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5e1dfb10-c355-4395-978b-c895bab566ed-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-z8vs7\" (UID: \"5e1dfb10-c355-4395-978b-c895bab566ed\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:40.230093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.230071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-telemeter-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.231605 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.231582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-federate-client-tls\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.235338 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.235296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s86f\" (UniqueName: \"kubernetes.io/projected/a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef-kube-api-access-5s86f\") pod \"telemeter-client-f67b6fd68-ll6wd\" (UID: \"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef\") " pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.311225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.311183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"
Jul 10 02:15:40.319132 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:40.318846    2579 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a34aaba98659bdef0020273533bcf144c418cb919cfd2da2413de63f4ded80de"
Jul 10 02:15:40.319132 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:15:40.318983    2579 kuberuntime_manager.go:1459] "Unhandled Error" err="container alertmanager start failed in pod alertmanager-main-0_openshift-monitoring(749eb7ac-4ddb-4da6-90ff-de53fee41af3): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 10 02:15:40.413161 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.413114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:15:40.646936 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:40.646841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7" event={"ID":"01c18038-ca2e-40c2-85ab-40cc7caa12e8","Type":"ContainerStarted","Data":"032c562505f95756327f7c41fe2b7255680c09c72d6da37f8c663691248de64c"}
Jul 10 02:15:41.034281 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.034238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:41.034281 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.034286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:41.037226 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.037202    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:15:41.038448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.038426    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:15:41.047922 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.047892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b8e300-2322-41e9-830d-0f0fe72835b8-original-pull-secret\") pod \"global-pull-secret-syncer-f5w9n\" (UID: \"38b8e300-2322-41e9-830d-0f0fe72835b8\") " pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:41.047922 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.047908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88c8b447-df78-4fb5-9932-c7154765ee06-metrics-certs\") pod \"network-metrics-daemon-7mx5l\" (UID: \"88c8b447-df78-4fb5-9932-c7154765ee06\") " pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:41.099376 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.098424    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:15:41.102048 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.102023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.107266 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.107240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-el3be52tfk2je\""
Jul 10 02:15:41.107422 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.107392    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.109261    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.109468    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.109679    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4s7lc\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.109946    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.110131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.110532    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 02:15:41.110898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.110757    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 02:15:41.115763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.114385    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 02:15:41.115763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.115152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 02:15:41.115763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.115306    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 02:15:41.115763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.115395    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 02:15:41.119758 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.117350    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 02:15:41.119758 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.117950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 02:15:41.124890 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.124448    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 02:15:41.126881 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.126861    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:15:41.135886 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.135772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:41.138788 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.138759    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:15:41.149000 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.148821    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:15:41.163518 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.163486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc8pc\" (UniqueName: \"kubernetes.io/projected/13777cb7-d2b1-4ff2-8e57-36af56759f4d-kube-api-access-lc8pc\") pod \"network-check-target-pkhrv\" (UID: \"13777cb7-d2b1-4ff2-8e57-36af56759f4d\") " pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:41.236891 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.236847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.236909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.236943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.236974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.236999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237309    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.237356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmk26\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.238006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.237535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.254812 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.254784    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-htmn4\""
Jul 10 02:15:41.258827 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.258803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5w9n"
Jul 10 02:15:41.263140 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.262944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7mx5l"
Jul 10 02:15:41.265945 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.265925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pmctl\""
Jul 10 02:15:41.273769 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.273723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:15:41.338994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.338915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.338994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.338970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.338998    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmk26\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339597    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.339877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.339762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.340487 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.340456    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.340602 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.340575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.340742 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.340705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.346610 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.346293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.349047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.348597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.349047 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.348750    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.350788 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.350762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.350880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.350823    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.351289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.351266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.351541 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.351516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.351673 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.351650    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.352221 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.352126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.353298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.352761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.353298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.352806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.353298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.353207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.354228 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.354147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmk26\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.354311 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.354288    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.354790 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.354766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.358381 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.357957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.358381 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.358333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config\") pod \"prometheus-k8s-0\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:41.420107 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:41.419878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:15:50.652061 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:50.649306    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"]
Jul 10 02:15:50.752802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:50.746405    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7" event={"ID":"5e1dfb10-c355-4395-978b-c895bab566ed","Type":"ContainerStarted","Data":"31758426e4dd8c13d774bb49e37d6962f5b5bed7754bd920e0debfc5df4964ea"}
Jul 10 02:15:50.881272 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:50.881246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f67b6fd68-ll6wd"]
Jul 10 02:15:50.889640 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:50.889610    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5eee088_9a6b_4c0f_8edb_dfe5f11aaeef.slice/crio-b217503f0dcc79b6df1915b342395f42d34497f9481450e74938fd2e4f864123 WatchSource:0}: Error finding container b217503f0dcc79b6df1915b342395f42d34497f9481450e74938fd2e4f864123: Status 404 returned error can't find the container with id b217503f0dcc79b6df1915b342395f42d34497f9481450e74938fd2e4f864123
Jul 10 02:15:50.900479 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:50.899447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pkhrv"]
Jul 10 02:15:51.000857 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.000427    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7mx5l"]
Jul 10 02:15:51.022708 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.022679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5w9n"]
Jul 10 02:15:51.036028 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.036005    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:15:51.039551 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:15:51.039523    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f512694_9a5d_4d06_9389_856032856ffb.slice/crio-8cb54121777dc59a91a43d7424dd754ff83547e1bdd9dfc93a7241c334693684 WatchSource:0}: Error finding container 8cb54121777dc59a91a43d7424dd754ff83547e1bdd9dfc93a7241c334693684: Status 404 returned error can't find the container with id 8cb54121777dc59a91a43d7424dd754ff83547e1bdd9dfc93a7241c334693684
Jul 10 02:15:51.782723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.781573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fbwrd" event={"ID":"1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb","Type":"ContainerStarted","Data":"c8a3776650b0caca9ddbb217aa6b85de8ef8958f7c83e1a8dabd47568bb8510f"}
Jul 10 02:15:51.782723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.782658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:51.787769 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.786868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd" event={"ID":"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef","Type":"ContainerStarted","Data":"b217503f0dcc79b6df1915b342395f42d34497f9481450e74938fd2e4f864123"}
Jul 10 02:15:51.789449 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.789382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pkhrv" event={"ID":"13777cb7-d2b1-4ff2-8e57-36af56759f4d","Type":"ContainerStarted","Data":"f1f352402517d42b1aadcf9997026f2ba351732fa3b0f7a8114c57647d46193b"}
Jul 10 02:15:51.796153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.795801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7" event={"ID":"01c18038-ca2e-40c2-85ab-40cc7caa12e8","Type":"ContainerStarted","Data":"761db6b8c39faf682dd15f775d8d168f69c22301c6004f1b7d244744251abe5e"}
Jul 10 02:15:51.797048 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.797032    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-fbwrd"
Jul 10 02:15:51.798474 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.798418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f5w9n" event={"ID":"38b8e300-2322-41e9-830d-0f0fe72835b8","Type":"ContainerStarted","Data":"06506a57d6f8b2fa0261f6eea028ec8ee5f82db48b8454596c76c3b9eef09b82"}
Jul 10 02:15:51.804693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.803628    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc" exitCode=0
Jul 10 02:15:51.804693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.803705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"}
Jul 10 02:15:51.804693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.803784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"8cb54121777dc59a91a43d7424dd754ff83547e1bdd9dfc93a7241c334693684"}
Jul 10 02:15:51.806551 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.806431    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-fbwrd" podStartSLOduration=2.402424383 podStartE2EDuration="24.806412447s" totalImagesPullingTime="22.403988064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:51.801899114 +0000 UTC m=+77.098480888" watchObservedRunningTime="2026-07-10 02:15:51.806412447 +0000 UTC m=+77.102994227"
Jul 10 02:15:51.818839 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.818786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccb89555-6j52s" event={"ID":"b43ede92-02c3-4e9e-b2f6-af06cce7820e","Type":"ContainerStarted","Data":"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"}
Jul 10 02:15:51.826144 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.826121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mx5l" event={"ID":"88c8b447-df78-4fb5-9932-c7154765ee06","Type":"ContainerStarted","Data":"0ec1100c9310fd8a4e1064dd2280e34faeb7a21f759ed2a228f4f7c98dfff795"}
Jul 10 02:15:51.834254 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.834228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"}
Jul 10 02:15:51.834351 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.834300    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"}
Jul 10 02:15:51.834351 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.834346    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"}
Jul 10 02:15:51.834461 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.834385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"}
Jul 10 02:15:51.839829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.839799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"7849ba988f3beb24f207a640582525ec9b8e3981385a5a24c575da93f1f19862"}
Jul 10 02:15:51.839990 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.839868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"46984947aefccee29c40eb6fb832284315f26e4616bcddab0ecd595b7480b679"}
Jul 10 02:15:51.839990 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.839910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"cd53fff3bb8051f79a9534d3096acb6be4d238e1d09116dd404aeaeccc03c2c6"}
Jul 10 02:15:51.845865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:51.844833    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7" podStartSLOduration=2.057503377 podStartE2EDuration="12.844815533s" totalImagesPullingTime="10.787312156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:51.823837974 +0000 UTC m=+77.120419744" watchObservedRunningTime="2026-07-10 02:15:51.844815533 +0000 UTC m=+77.141397310"
Jul 10 02:15:55.157549 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:55.157491    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75ccb89555-6j52s" podStartSLOduration=6.213201498 podStartE2EDuration="19.157467188s" totalImagesPullingTime="12.94426569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:51.891247677 +0000 UTC m=+77.187829454" watchObservedRunningTime="2026-07-10 02:15:55.157467188 +0000 UTC m=+80.454048963"
Jul 10 02:15:55.159658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:55.159628    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:15:57.092776 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:57.092723    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_749eb7ac-4ddb-4da6-90ff-de53fee41af3/init-config-reloader/0.log"
Jul 10 02:15:57.095602 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:57.095541    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:15:58.379078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:58.379045    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-state-metrics/0.log"
Jul 10 02:15:58.578431 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:58.578400    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-rbac-proxy-main/0.log"
Jul 10 02:15:58.778866 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:58.778838    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-rbac-proxy-self/0.log"
Jul 10 02:15:58.978227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:58.978176    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7dff7d7994-dkkl7_01c18038-ca2e-40c2-85ab-40cc7caa12e8/metrics-server/0.log"
Jul 10 02:15:59.378717 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:59.378681    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/init-textfile/0.log"
Jul 10 02:15:59.579040 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:59.579001    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/node-exporter/0.log"
Jul 10 02:15:59.583654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:59.583612    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:59.583780 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:59.583693    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:15:59.778268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:15:59.778240    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/kube-rbac-proxy/0.log"
Jul 10 02:16:01.178893 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:01.178808    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/kube-rbac-proxy-main/0.log"
Jul 10 02:16:01.378228 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:01.378172    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/kube-rbac-proxy-self/0.log"
Jul 10 02:16:01.578907 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:01.578885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/openshift-state-metrics/0.log"
Jul 10 02:16:01.781578 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:01.781560    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f512694-9a5d-4d06-9389-856032856ffb/init-config-reloader/0.log"
Jul 10 02:16:01.893978 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:01.893487    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"alertmanager\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3"
Jul 10 02:16:01.897193 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:01.897169    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"}
Jul 10 02:16:02.905792 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.905703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mx5l" event={"ID":"88c8b447-df78-4fb5-9932-c7154765ee06","Type":"ContainerStarted","Data":"05febd38aa78deff5d87ac8d5aa6ca18d1462aaa1dfb498a57d2407dfea26d2d"}
Jul 10 02:16:02.906300 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.905818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7mx5l" event={"ID":"88c8b447-df78-4fb5-9932-c7154765ee06","Type":"ContainerStarted","Data":"ed3bd4818f3ce407b1621d80a4569a305728b51c9fe72b762b162c252e825616"}
Jul 10 02:16:02.908653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.908628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"db9ffcb77fa91ae8a69b6db75015a2cf48dd57d69331cf65044ea3adf16a3bb8"}
Jul 10 02:16:02.908807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.908681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"6140394878e7e2971bd27408ba423a02482e95643054cb402aade474780807d9"}
Jul 10 02:16:02.908807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.908706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" event={"ID":"521a4d1c-c832-4ecb-a552-4ea144f712d0","Type":"ContainerStarted","Data":"aeb4a0fb9ac27d197639a7ce1640e41697e1028e7b67bd2b0069e8abc28980b0"}
Jul 10 02:16:02.908807 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.908767    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:16:02.910474 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.910446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7" event={"ID":"5e1dfb10-c355-4395-978b-c895bab566ed","Type":"ContainerStarted","Data":"d980b230d8d335d25f3d5c4c8f99586f8e68064fa6dfafadc264b933f27f0ebb"}
Jul 10 02:16:02.910690 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.910663    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:16:02.914503 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.914477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd" event={"ID":"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef","Type":"ContainerStarted","Data":"6ada9bee3fa6c0b65d5db31addbd452d1bbba49637435ef696e7fead614799e3"}
Jul 10 02:16:02.914612 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.914533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd" event={"ID":"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef","Type":"ContainerStarted","Data":"a983e020a4ed156008e3f8c624ac33d49b105a6088d44f8dd8171312cb59fb09"}
Jul 10 02:16:02.914612 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.914563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd" event={"ID":"a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef","Type":"ContainerStarted","Data":"4a0cd49e4ef6adf5d74edf878fdf1b8e1536467488dbbf3b7ae22771be873818"}
Jul 10 02:16:02.916402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.916382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7"
Jul 10 02:16:02.916769 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.916718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pkhrv" event={"ID":"13777cb7-d2b1-4ff2-8e57-36af56759f4d","Type":"ContainerStarted","Data":"6e1369b823fde8f7c8bd9bc7de34338a3ebe807bf48620a5e8ba93a4340bd6b0"}
Jul 10 02:16:02.916888 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.916847    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl"
Jul 10 02:16:02.916959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.916900    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:16:02.918183 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.918166    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f5w9n" event={"ID":"38b8e300-2322-41e9-830d-0f0fe72835b8","Type":"ContainerStarted","Data":"02e0374eaae472ded57addd2536b54f354aeb4eb838e5db9567ca9673f9eb85b"}
Jul 10 02:16:02.921407 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"}
Jul 10 02:16:02.921494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"}
Jul 10 02:16:02.921494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"}
Jul 10 02:16:02.921577 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"}
Jul 10 02:16:02.921577 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921525    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"}
Jul 10 02:16:02.921577 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.921552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerStarted","Data":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"}
Jul 10 02:16:02.925259 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.925216    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7mx5l" podStartSLOduration=76.190540281 podStartE2EDuration="1m26.925199293s" totalImagesPullingTime="10.734659012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:02.924405955 +0000 UTC m=+88.220987730" watchObservedRunningTime="2026-07-10 02:16:02.925199293 +0000 UTC m=+88.221781069"
Jul 10 02:16:02.943532 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.943488    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f5w9n" podStartSLOduration=76.221828437 podStartE2EDuration="1m26.94346348s" totalImagesPullingTime="10.721635043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:02.943032251 +0000 UTC m=+88.239614026" watchObservedRunningTime="2026-07-10 02:16:02.94346348 +0000 UTC m=+88.240045254"
Jul 10 02:16:02.966328 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.966172    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65bfd8447f-g69pl" podStartSLOduration=3.023681143 podStartE2EDuration="25.966154136s" totalImagesPullingTime="22.942472993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:02.965572512 +0000 UTC m=+88.262154299" watchObservedRunningTime="2026-07-10 02:16:02.966154136 +0000 UTC m=+88.262735912"
Jul 10 02:16:02.978820 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.978797    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8f512694-9a5d-4d06-9389-856032856ffb/kube-rbac-proxy-thanos/0.log"
Jul 10 02:16:02.989963 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:02.989893    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f67b6fd68-ll6wd" podStartSLOduration=13.141052796 podStartE2EDuration="23.989872202s" totalImagesPullingTime="10.848819406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:02.988390209 +0000 UTC m=+88.284971984" watchObservedRunningTime="2026-07-10 02:16:02.989872202 +0000 UTC m=+88.286453977"
Jul 10 02:16:03.011399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.011346    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pkhrv" podStartSLOduration=76.171527199 podStartE2EDuration="1m27.011329485s" totalImagesPullingTime="10.839802286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:03.01058694 +0000 UTC m=+88.307168715" watchObservedRunningTime="2026-07-10 02:16:03.011329485 +0000 UTC m=+88.307911260"
Jul 10 02:16:03.054938 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.054882    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-z8vs7" podStartSLOduration=13.963043634 podStartE2EDuration="24.054863754s" totalImagesPullingTime="10.09182012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:03.054775471 +0000 UTC m=+88.351357247" watchObservedRunningTime="2026-07-10 02:16:03.054863754 +0000 UTC m=+88.351445527"
Jul 10 02:16:03.086540 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.086492    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=12.114329265 podStartE2EDuration="22.086476064s" totalImagesPullingTime="9.950402866s" totalInitContainerRuntime="21.743933ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:03.08629756 +0000 UTC m=+88.382879337" watchObservedRunningTime="2026-07-10 02:16:03.086476064 +0000 UTC m=+88.383057839"
Jul 10 02:16:03.179429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.179335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rrpkh_c1f85e19-00b6-4576-bd54-afca38e59c15/prometheus-operator/0.log"
Jul 10 02:16:03.378453 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.378420    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rrpkh_c1f85e19-00b6-4576-bd54-afca38e59c15/kube-rbac-proxy/0.log"
Jul 10 02:16:03.579401 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.579372    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-pvphl_2f3ff963-ff77-447a-8642-42edd4a71ef3/prometheus-operator-admission-webhook/0.log"
Jul 10 02:16:03.779153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.779121    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/telemeter-client/0.log"
Jul 10 02:16:03.978395 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:03.978311    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/reload/0.log"
Jul 10 02:16:04.178475 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:04.178444    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/kube-rbac-proxy/0.log"
Jul 10 02:16:04.378472 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:04.378437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/thanos-query/0.log"
Jul 10 02:16:04.578335 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:04.578297    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-web/0.log"
Jul 10 02:16:04.778779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:04.778755    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy/0.log"
Jul 10 02:16:04.978061 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:04.978044    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/prom-label-proxy/0.log"
Jul 10 02:16:05.179012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:05.178974    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-rules/0.log"
Jul 10 02:16:05.378671 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:05.378598    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-metrics/0.log"
Jul 10 02:16:05.938659 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:05.938624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerStarted","Data":"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"}
Jul 10 02:16:05.970748 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:05.970675    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.185792858 podStartE2EDuration="30.970652695s" totalImagesPullingTime="26.759089549s" totalInitContainerRuntime="25.770288ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-10 02:16:05.968437477 +0000 UTC m=+91.265019253" watchObservedRunningTime="2026-07-10 02:16:05.970652695 +0000 UTC m=+91.267234471"
Jul 10 02:16:06.177972 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:06.177938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75ccb89555-6j52s_b43ede92-02c3-4e9e-b2f6-af06cce7820e/console/0.log"
Jul 10 02:16:06.420530 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:06.420498    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:16:06.579179 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:06.579143    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fbwrd_1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb/download-server/0.log"
Jul 10 02:16:19.588673 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:19.588641    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:16:19.592747 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:19.592709    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7dff7d7994-dkkl7"
Jul 10 02:16:20.200114 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.200048    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75ccb89555-6j52s" podUID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" containerName="console" containerID="cri-o://7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01" gracePeriod=15
Jul 10 02:16:20.497238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.497204    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:16:20.649973 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.649936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.649973 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.649978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650045    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dslkw\" (UniqueName: \"kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw\") pod \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\" (UID: \"b43ede92-02c3-4e9e-b2f6-af06cce7820e\") "
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650423    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config" (OuterVolumeSpecName: "console-config") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:20.650502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650440    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:20.650766 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.650535    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca" (OuterVolumeSpecName: "service-ca") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:20.652499 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.652477    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw" (OuterVolumeSpecName: "kube-api-access-dslkw") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "kube-api-access-dslkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:20.652499 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.652484    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:20.652601 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.652544    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b43ede92-02c3-4e9e-b2f6-af06cce7820e" (UID: "b43ede92-02c3-4e9e-b2f6-af06cce7820e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750864    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750891    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750901    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dslkw\" (UniqueName: \"kubernetes.io/projected/b43ede92-02c3-4e9e-b2f6-af06cce7820e-kube-api-access-dslkw\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750911    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-oauth-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750920    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-oauth-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:20.750943 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:20.750927    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b43ede92-02c3-4e9e-b2f6-af06cce7820e-console-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:21.007785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.007675    2579 generic.go:356] "Generic (PLEG): container finished" podID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" containerID="7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01" exitCode=0
Jul 10 02:16:21.007785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.007745    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccb89555-6j52s"
Jul 10 02:16:21.007785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.007768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccb89555-6j52s" event={"ID":"b43ede92-02c3-4e9e-b2f6-af06cce7820e","Type":"ContainerDied","Data":"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"}
Jul 10 02:16:21.008028 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.007840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccb89555-6j52s" event={"ID":"b43ede92-02c3-4e9e-b2f6-af06cce7820e","Type":"ContainerDied","Data":"af1774fcfdcbabe959223207855144a71d01225c1c99e4ec0353f821d7860a35"}
Jul 10 02:16:21.008028 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.007861    2579 scope.go:117] "RemoveContainer" containerID="7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"
Jul 10 02:16:21.018582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.018565    2579 scope.go:117] "RemoveContainer" containerID="7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"
Jul 10 02:16:21.018853 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:21.018835    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01\": container with ID starting with 7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01 not found: ID does not exist" containerID="7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"
Jul 10 02:16:21.018935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.018861    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01"} err="failed to get container status \"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01\": rpc error: code = NotFound desc = could not find container \"7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01\": container with ID starting with 7dcddd3dec9270d211b31e10aed90394680ef02fccf925c415480e6cdf4f3b01 not found: ID does not exist"
Jul 10 02:16:21.029259 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.029237    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:16:21.032916 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.032897    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75ccb89555-6j52s"]
Jul 10 02:16:21.244302 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:21.244262    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" path="/var/lib/kubelet/pods/b43ede92-02c3-4e9e-b2f6-af06cce7820e/volumes"
Jul 10 02:16:33.930817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:33.930781    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pkhrv"
Jul 10 02:16:41.420593 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:41.420558    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:16:41.476659 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:41.476631    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:16:42.110681 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:42.110658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:16:55.248216 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248186    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248553    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="config-reloader" containerID="cri-o://03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe" gracePeriod=120
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248640    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-metric" containerID="cri-o://7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728" gracePeriod=120
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248664    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="alertmanager" containerID="cri-o://cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60" gracePeriod=120
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248619    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="prom-label-proxy" containerID="cri-o://13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52" gracePeriod=120
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248623    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy" containerID="cri-o://c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81" gracePeriod=120
Jul 10 02:16:55.248815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:55.248643    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-web" containerID="cri-o://339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5" gracePeriod=120
Jul 10 02:16:56.152599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152568    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60" exitCode=0
Jul 10 02:16:56.152599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152593    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52" exitCode=0
Jul 10 02:16:56.152599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152600    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728" exitCode=0
Jul 10 02:16:56.152599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152605    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81" exitCode=0
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152610    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe" exitCode=0
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"}
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152697    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"}
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"}
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"}
Jul 10 02:16:56.152874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.152786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"}
Jul 10 02:16:56.490703 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.490681    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:56.587167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587135    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587205    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587226    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587257    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587276    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5fw9\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587324    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587373    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587399    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587445    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587472    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587504    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587528    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config\") pod \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\" (UID: \"749eb7ac-4ddb-4da6-90ff-de53fee41af3\") "
Jul 10 02:16:56.587635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587539    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:16:56.587982 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:56.587982 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587864    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.587982 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.587885    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-alertmanager-main-db\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.588352 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.588253    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:56.591362 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591325    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.591470 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591353    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:56.591531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591500    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume" (OuterVolumeSpecName: "config-volume") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.591798 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591758    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.591912 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591785    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out" (OuterVolumeSpecName: "config-out") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:16:56.591994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.591964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.592075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.592051    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9" (OuterVolumeSpecName: "kube-api-access-t5fw9") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "kube-api-access-t5fw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:56.592576 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.592555    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.595758 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.595638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.602623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.602600    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config" (OuterVolumeSpecName: "web-config") pod "749eb7ac-4ddb-4da6-90ff-de53fee41af3" (UID: "749eb7ac-4ddb-4da6-90ff-de53fee41af3"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689105    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-main-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689150    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-volume\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689162    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689192    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-web-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689202    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-cluster-tls-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689211    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-tls-assets\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689219    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5fw9\" (UniqueName: \"kubernetes.io/projected/749eb7ac-4ddb-4da6-90ff-de53fee41af3-kube-api-access-t5fw9\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689227    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/749eb7ac-4ddb-4da6-90ff-de53fee41af3-config-out\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689235    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689244    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/749eb7ac-4ddb-4da6-90ff-de53fee41af3-metrics-client-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:56.689496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:56.689252    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/749eb7ac-4ddb-4da6-90ff-de53fee41af3-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:57.160113 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.160074    2579 generic.go:356] "Generic (PLEG): container finished" podID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerID="339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5" exitCode=0
Jul 10 02:16:57.160289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.160124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"}
Jul 10 02:16:57.160289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.160166    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.160289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.160203    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"749eb7ac-4ddb-4da6-90ff-de53fee41af3","Type":"ContainerDied","Data":"5e63c432f84630283272f62ff884f64b4fa7e7509ac4644adf1abd59f85ff3c9"}
Jul 10 02:16:57.160289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.160252    2579 scope.go:117] "RemoveContainer" containerID="cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"
Jul 10 02:16:57.170037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.170014    2579 scope.go:117] "RemoveContainer" containerID="13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"
Jul 10 02:16:57.177141 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.177123    2579 scope.go:117] "RemoveContainer" containerID="7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"
Jul 10 02:16:57.184025 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.184001    2579 scope.go:117] "RemoveContainer" containerID="c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"
Jul 10 02:16:57.186276 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.186256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:57.193123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.193095    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:57.197391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.197373    2579 scope.go:117] "RemoveContainer" containerID="339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"
Jul 10 02:16:57.204430 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.204411    2579 scope.go:117] "RemoveContainer" containerID="03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"
Jul 10 02:16:57.211983 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.211965    2579 scope.go:117] "RemoveContainer" containerID="170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598"
Jul 10 02:16:57.217134 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217113    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:57.217435 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217423    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" containerName="console"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217437    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" containerName="console"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217452    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="prom-label-proxy"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217457    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="prom-label-proxy"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217467    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-web"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217473    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-web"
Jul 10 02:16:57.217478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217478    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-metric"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217484    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-metric"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217494    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="init-config-reloader"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217499    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="init-config-reloader"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217508    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217513    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217520    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="config-reloader"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217525    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="config-reloader"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217530    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="alertmanager"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217535    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="alertmanager"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217579    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="config-reloader"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217586    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-metric"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217592    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="prom-label-proxy"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217598    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b43ede92-02c3-4e9e-b2f6-af06cce7820e" containerName="console"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217604    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="alertmanager"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217610    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy-web"
Jul 10 02:16:57.217665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.217614    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" containerName="kube-rbac-proxy"
Jul 10 02:16:57.219452 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.219435    2579 scope.go:117] "RemoveContainer" containerID="cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"
Jul 10 02:16:57.219674 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.219658    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60\": container with ID starting with cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60 not found: ID does not exist" containerID="cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"
Jul 10 02:16:57.219718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.219681    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60"} err="failed to get container status \"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60\": rpc error: code = NotFound desc = could not find container \"cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60\": container with ID starting with cbecf84ca016f11c326c28479eccd7f39a2bc42c5c53f6bd398abf573e5edd60 not found: ID does not exist"
Jul 10 02:16:57.219718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.219697    2579 scope.go:117] "RemoveContainer" containerID="13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"
Jul 10 02:16:57.219984 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.219963    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52\": container with ID starting with 13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52 not found: ID does not exist" containerID="13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"
Jul 10 02:16:57.220035 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.219991    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52"} err="failed to get container status \"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52\": rpc error: code = NotFound desc = could not find container \"13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52\": container with ID starting with 13d30dd11a1eb6d93f195eb4594967e3e5560602fc16e26f882ed4046f27dd52 not found: ID does not exist"
Jul 10 02:16:57.220035 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.220005    2579 scope.go:117] "RemoveContainer" containerID="7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"
Jul 10 02:16:57.220274 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.220256    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728\": container with ID starting with 7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728 not found: ID does not exist" containerID="7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"
Jul 10 02:16:57.220311 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.220281    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728"} err="failed to get container status \"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728\": rpc error: code = NotFound desc = could not find container \"7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728\": container with ID starting with 7d9ff6bb662e04c181bd5cfd47698bf02813ca493b4dfe7fa1ca04431db30728 not found: ID does not exist"
Jul 10 02:16:57.220311 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.220297    2579 scope.go:117] "RemoveContainer" containerID="c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"
Jul 10 02:16:57.220546 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.220531    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81\": container with ID starting with c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81 not found: ID does not exist" containerID="c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"
Jul 10 02:16:57.220607 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.220549    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81"} err="failed to get container status \"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81\": rpc error: code = NotFound desc = could not find container \"c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81\": container with ID starting with c4a45e85e8b80df937ba56a3b8a3154d57646a0602fb0b126693e9f80be96f81 not found: ID does not exist"
Jul 10 02:16:57.220950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.220562    2579 scope.go:117] "RemoveContainer" containerID="339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"
Jul 10 02:16:57.221163 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.221224 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.221194    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5\": container with ID starting with 339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5 not found: ID does not exist" containerID="339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"
Jul 10 02:16:57.221224 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221214    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5"} err="failed to get container status \"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5\": rpc error: code = NotFound desc = could not find container \"339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5\": container with ID starting with 339b5f44ca0f01ee8e9c46d98b9233ea3c89d48195cf8387d2e7e44cc29c19a5 not found: ID does not exist"
Jul 10 02:16:57.221312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221228    2579 scope.go:117] "RemoveContainer" containerID="03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"
Jul 10 02:16:57.221512 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.221478    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe\": container with ID starting with 03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe not found: ID does not exist" containerID="03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"
Jul 10 02:16:57.221592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221514    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe"} err="failed to get container status \"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe\": rpc error: code = NotFound desc = could not find container \"03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe\": container with ID starting with 03bc63f2bbf15dae4d06ec2dd0b13fc27313bb75b728f19899e1610f3339f5fe not found: ID does not exist"
Jul 10 02:16:57.221592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221536    2579 scope.go:117] "RemoveContainer" containerID="170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598"
Jul 10 02:16:57.221824 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:16:57.221807    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598\": container with ID starting with 170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598 not found: ID does not exist" containerID="170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598"
Jul 10 02:16:57.221879 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.221827    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598"} err="failed to get container status \"170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598\": rpc error: code = NotFound desc = could not find container \"170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598\": container with ID starting with 170798dd514f6a2ce8e34b618d8f7d4f340b6dc0328831569d678864b67cf598 not found: ID does not exist"
Jul 10 02:16:57.224089 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 02:16:57.224089 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224069    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 02:16:57.224243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224070    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 02:16:57.224243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224133    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 02:16:57.224243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224149    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 02:16:57.224243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224079    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mns7f\""
Jul 10 02:16:57.224482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224472    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 02:16:57.224556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224538    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 02:16:57.224556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.224471    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 02:16:57.230558 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.230540    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 02:16:57.234610 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.234593    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:57.244225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.244205    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749eb7ac-4ddb-4da6-90ff-de53fee41af3" path="/var/lib/kubelet/pods/749eb7ac-4ddb-4da6-90ff-de53fee41af3/volumes"
Jul 10 02:16:57.396944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.396915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.396958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.396981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-out\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.396999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-web-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6zxf\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-kube-api-access-w6zxf\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397486 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397486 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.397486 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.397385    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.497880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6zxf\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-kube-api-access-w6zxf\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.497880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.497880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.497880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.497927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-out\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-web-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.498457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.499104 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.498501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.499158 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.499135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.499451 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.499427    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501098 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501211 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501273 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501477 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501597 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-web-config\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.501788 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.501768    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-volume\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.502397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.502371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.502571 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.502556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-config-out\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.503276 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.503260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.506594 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.506577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6zxf\" (UniqueName: \"kubernetes.io/projected/80c524ec-e303-43c4-b224-9a7a8f1ac2c8-kube-api-access-w6zxf\") pod \"alertmanager-main-0\" (UID: \"80c524ec-e303-43c4-b224-9a7a8f1ac2c8\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.533094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.533076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 02:16:57.668951 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:57.668930    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 02:16:57.670940 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:16:57.670914    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c524ec_e303_43c4_b224_9a7a8f1ac2c8.slice/crio-2cd00d10b6b7356503479ca373c8530f134c2ebc0422d8be39e9df56ed6270e6 WatchSource:0}: Error finding container 2cd00d10b6b7356503479ca373c8530f134c2ebc0422d8be39e9df56ed6270e6: Status 404 returned error can't find the container with id 2cd00d10b6b7356503479ca373c8530f134c2ebc0422d8be39e9df56ed6270e6
Jul 10 02:16:58.165823 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:58.165788    2579 generic.go:356] "Generic (PLEG): container finished" podID="80c524ec-e303-43c4-b224-9a7a8f1ac2c8" containerID="14351f67f6f037bc058e1cce561a54b334ca21d24a684fa9dfba5bdb32781362" exitCode=0
Jul 10 02:16:58.165995 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:58.165879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerDied","Data":"14351f67f6f037bc058e1cce561a54b334ca21d24a684fa9dfba5bdb32781362"}
Jul 10 02:16:58.165995 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:58.165947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"2cd00d10b6b7356503479ca373c8530f134c2ebc0422d8be39e9df56ed6270e6"}
Jul 10 02:16:59.174999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.174963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"c693a517871b07cac625cf482c2e8506490da9141cab63f1cc71a8642eddf593"}
Jul 10 02:16:59.175390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.175031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"44892a7e29da9839aabcf2d5202ea68aa0f3b795e9af2855688ea01e8e902af2"}
Jul 10 02:16:59.175390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.175063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"85026e55b4c879fa88463a03d79c74bb3869ea7ddb30cce653026c4370884855"}
Jul 10 02:16:59.175390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.175090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"101a013346bbb78702834766d0d3777e98da69facea65447ccfd49223f2db3e2"}
Jul 10 02:16:59.175390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.175123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"7d2ef33fd75629110124d1a11336643514dfb95082dd7dcb79cad43169a4a7ef"}
Jul 10 02:16:59.175390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.175147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"80c524ec-e303-43c4-b224-9a7a8f1ac2c8","Type":"ContainerStarted","Data":"8f79cf3c90a6a0477a551379b457d332d8a7742da52be583e022aab73036adcf"}
Jul 10 02:16:59.203775 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.203713    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.187861226 podStartE2EDuration="2.203697498s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.836272ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:59.201813775 +0000 UTC m=+144.498395549" watchObservedRunningTime="2026-07-10 02:16:59.203697498 +0000 UTC m=+144.500279274"
Jul 10 02:16:59.468563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468478    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468880    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="prometheus" containerID="cri-o://b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5" gracePeriod=600
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468894    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy" containerID="cri-o://299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41" gracePeriod=600
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468930    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4" gracePeriod=600
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468936    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-web" containerID="cri-o://131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e" gracePeriod=600
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468903    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="thanos-sidecar" containerID="cri-o://6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a" gracePeriod=600
Jul 10 02:16:59.469051 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.468910    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="config-reloader" containerID="cri-o://56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122" gracePeriod=600
Jul 10 02:16:59.728915 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.728855    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:16:59.818255 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818262    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818298    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818337    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818411    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818451    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818527    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818561    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818597    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818620    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818643    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818698    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.818799 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmk26\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.819418 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2\") pod \"8f512694-9a5d-4d06-9389-856032856ffb\" (UID: \"8f512694-9a5d-4d06-9389-856032856ffb\") "
Jul 10 02:16:59.819418 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.818926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.819418 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.819139    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.819418 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.819156    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.819418 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.819252    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:16:59.819657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.819487    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.822118 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.820317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.822118 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.820350    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.822118 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.820709    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.822118 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.822082    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:59.822442 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.822397    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.822853 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.822820    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.822975 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.822949    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.823143 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.823123    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.823254 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.823217    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:59.823367 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.823344    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.824236 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.824203    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26" (OuterVolumeSpecName: "kube-api-access-lmk26") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "kube-api-access-lmk26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:59.824369 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.824351    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config" (OuterVolumeSpecName: "config") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.824885 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.824826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.825192 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.825101    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.827129 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.827104    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out" (OuterVolumeSpecName: "config-out") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:16:59.835694 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.835673    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config" (OuterVolumeSpecName: "web-config") pod "8f512694-9a5d-4d06-9389-856032856ffb" (UID: "8f512694-9a5d-4d06-9389-856032856ffb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:59.920045 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920019    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-db\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920045 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920044    2579 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920055    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8f512694-9a5d-4d06-9389-856032856ffb-config-out\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920064    2579 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-metrics-client-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920074    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-tls-assets\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920082    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920091    2579 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-configmap-metrics-client-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920099    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920108    2579 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920117    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-web-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920124    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920133    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920141    2579 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-grpc-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920149    2579 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920156    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920166    2579 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8f512694-9a5d-4d06-9389-856032856ffb-secret-kube-rbac-proxy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920175    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmk26\" (UniqueName: \"kubernetes.io/projected/8f512694-9a5d-4d06-9389-856032856ffb-kube-api-access-lmk26\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:16:59.920217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:16:59.920183    2579 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8f512694-9a5d-4d06-9389-856032856ffb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182607    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4" exitCode=0
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182630    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41" exitCode=0
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182637    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e" exitCode=0
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182642    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a" exitCode=0
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182647    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122" exitCode=0
Jul 10 02:17:00.182636 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182652    2579 generic.go:356] "Generic (PLEG): container finished" podID="8f512694-9a5d-4d06-9389-856032856ffb" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5" exitCode=0
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182777    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.182997    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.183019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.183059    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.183099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"}
Jul 10 02:17:00.183268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.183136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8f512694-9a5d-4d06-9389-856032856ffb","Type":"ContainerDied","Data":"8cb54121777dc59a91a43d7424dd754ff83547e1bdd9dfc93a7241c334693684"}
Jul 10 02:17:00.191637 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.191521    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.198894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.198878    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.206174 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.206159    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.208426 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.208394    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:17:00.213604 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.213582    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:17:00.214173 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.214159    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.220880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.220864    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.228139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.228109    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.234996 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.234980    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.235249 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.235231    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.235321 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235258    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.235321 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235276    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.235520 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.235503    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.235557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235528    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.235557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235543    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.235848 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.235832    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.235891 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235853    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.235891 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.235867    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.236097 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.236083    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.236166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236100    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.236166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236111    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.236332 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.236309    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.236370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236334    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.236370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236346    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.236542 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.236527    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.236586 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236545    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.236586 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236558    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.236780 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:17:00.236764    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.236829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236782    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.236829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236793    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.236980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236964    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.237017 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.236980    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.237154 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237139    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.237195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237154    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.237342 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237327    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.237386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237342    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.237521 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237506    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.237556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237522    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.237711 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237697    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.237770 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237711    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.237898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237882    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.237940 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.237899    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.238100 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238084    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.238137 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238100    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.238262 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238247    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.238301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238262    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.238437 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238422    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.238487 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238437    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.238599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238584    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.238651 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238600    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.238837 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238820    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.238837 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.238837    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.239154 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.239099    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.239254 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.239170    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.240107 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.240078    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.240207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.240108    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.240972 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.240948    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.240972 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.240973    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.241262 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241239    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.241331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241265    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.241531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241513    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.241578 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241533    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.241779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241758    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.241779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.241778    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.242083 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242029    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:17:00.242083 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242028    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.242083 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242055    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.242350 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242328    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.242417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242350    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.242512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242467    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-web"
Jul 10 02:17:00.242512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242489    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-web"
Jul 10 02:17:00.242512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242508    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="init-config-reloader"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242543    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="init-config-reloader"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242553    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="prometheus"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242561    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="prometheus"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242574    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242593    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242607    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="config-reloader"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="config-reloader"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242623    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-thanos"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242622    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242642    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-thanos"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242710    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="thanos-sidecar"
Jul 10 02:17:00.242759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242720    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="thanos-sidecar"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242852    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-thanos"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242870    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242881    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="prometheus"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242891    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="config-reloader"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242899    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="thanos-sidecar"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.242912    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f512694-9a5d-4d06-9389-856032856ffb" containerName="kube-rbac-proxy-web"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243010    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.243360 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243028    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.243704 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243357    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.243704 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243383    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.243704 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243619    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.243704 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243641    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.244011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243936    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.244011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.243960    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244174    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244197    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244421    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244443    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244641    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.244662    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245014    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.245075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245039    2579 scope.go:117] "RemoveContainer" containerID="94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"
Jul 10 02:17:00.245469 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245330    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4"} err="failed to get container status \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": rpc error: code = NotFound desc = could not find container \"94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4\": container with ID starting with 94904c8cf9786edb9be79e9f321813018781d4aa305cb4c859fd2d91fb4e4ab4 not found: ID does not exist"
Jul 10 02:17:00.245469 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245353    2579 scope.go:117] "RemoveContainer" containerID="299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"
Jul 10 02:17:00.245649 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245612    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41"} err="failed to get container status \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": rpc error: code = NotFound desc = could not find container \"299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41\": container with ID starting with 299a0722ac875ba42c2bcd9745b0ee1111b8f382d6b7b90f00c22f4280750a41 not found: ID does not exist"
Jul 10 02:17:00.245649 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245636    2579 scope.go:117] "RemoveContainer" containerID="131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"
Jul 10 02:17:00.245864 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245843    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e"} err="failed to get container status \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": rpc error: code = NotFound desc = could not find container \"131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e\": container with ID starting with 131445e33383498079449e2889b13b537b46a996bd26605afe295a08fa40eb9e not found: ID does not exist"
Jul 10 02:17:00.245917 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.245867    2579 scope.go:117] "RemoveContainer" containerID="6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"
Jul 10 02:17:00.246140 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246119    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a"} err="failed to get container status \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": rpc error: code = NotFound desc = could not find container \"6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a\": container with ID starting with 6a13c6a9cabc475b4fbbb3825326fb0bb501a508768052729f74dab7ac11355a not found: ID does not exist"
Jul 10 02:17:00.246193 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246141    2579 scope.go:117] "RemoveContainer" containerID="56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"
Jul 10 02:17:00.246438 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246411    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122"} err="failed to get container status \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": rpc error: code = NotFound desc = could not find container \"56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122\": container with ID starting with 56868743f8a3b37906295ff5e807e151ea80ab3d6c1612aeba4c4700ce855122 not found: ID does not exist"
Jul 10 02:17:00.246438 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246433    2579 scope.go:117] "RemoveContainer" containerID="b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"
Jul 10 02:17:00.246683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246663    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5"} err="failed to get container status \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": rpc error: code = NotFound desc = could not find container \"b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5\": container with ID starting with b003d180362c71aa41f986d4ebe6341b2533ed71ddd3a2e9098242e8a6c5f6e5 not found: ID does not exist"
Jul 10 02:17:00.246768 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246683    2579 scope.go:117] "RemoveContainer" containerID="0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"
Jul 10 02:17:00.246984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.246965    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc"} err="failed to get container status \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": rpc error: code = NotFound desc = could not find container \"0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc\": container with ID starting with 0a18b6caeeae8edcb7cdddd3480aae03af5a10d75fcf2341c1eff6c170ec48cc not found: ID does not exist"
Jul 10 02:17:00.248186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.248168    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.251059 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251030    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 02:17:00.251166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 02:17:00.251166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251030    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-el3be52tfk2je\""
Jul 10 02:17:00.251166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251103    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 02:17:00.251166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251034    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4s7lc\""
Jul 10 02:17:00.251166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251062    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 02:17:00.251600 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251493    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 02:17:00.251600 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251503    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 02:17:00.251600 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251509    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 02:17:00.251600 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 02:17:00.252314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251618    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 02:17:00.252314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 02:17:00.252314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251712    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 02:17:00.252314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.251789    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 02:17:00.255171 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.255139    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 02:17:00.256993 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.256977    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 02:17:00.260991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.260970    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:17:00.425269 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425269 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wnhz\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-kube-api-access-9wnhz\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425716    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.425838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.425830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526684 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526690    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wnhz\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-kube-api-access-9wnhz\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526920 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526920 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.526920 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526903    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.526977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527369 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527339    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.527686 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.528087 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.528087 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.528087 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.527815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.529841 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.529819    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.529952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.529859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a51ba4c-143a-427f-825c-060de64f6c11-config-out\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530967 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530768    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530967 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.530967 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.530824    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.531706 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.531407    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.531706 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.531636    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-web-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.531706 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.531649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.532448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.532424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.532448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.532444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.533075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.533055    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.533284 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.533265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-config\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.533770 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.533747    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6a51ba4c-143a-427f-825c-060de64f6c11-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.534260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.534239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6a51ba4c-143a-427f-825c-060de64f6c11-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.535092 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.535072    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wnhz\" (UniqueName: \"kubernetes.io/projected/6a51ba4c-143a-427f-825c-060de64f6c11-kube-api-access-9wnhz\") pod \"prometheus-k8s-0\" (UID: \"6a51ba4c-143a-427f-825c-060de64f6c11\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.558174 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.558156    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:17:00.699639 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:00.699605    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 02:17:00.702439 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:17:00.702416    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a51ba4c_143a_427f_825c_060de64f6c11.slice/crio-97ec56d3063ff17759867c36ab510878eb9d044bcb9624ddd2da6abe504efb15 WatchSource:0}: Error finding container 97ec56d3063ff17759867c36ab510878eb9d044bcb9624ddd2da6abe504efb15: Status 404 returned error can't find the container with id 97ec56d3063ff17759867c36ab510878eb9d044bcb9624ddd2da6abe504efb15
Jul 10 02:17:01.187894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:01.187861    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a51ba4c-143a-427f-825c-060de64f6c11" containerID="586f978f63c004e8e6b67fad645e7d76807fb936a20decdfc34afb6973ece048" exitCode=0
Jul 10 02:17:01.188344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:01.187960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerDied","Data":"586f978f63c004e8e6b67fad645e7d76807fb936a20decdfc34afb6973ece048"}
Jul 10 02:17:01.188344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:01.188083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"97ec56d3063ff17759867c36ab510878eb9d044bcb9624ddd2da6abe504efb15"}
Jul 10 02:17:01.244220 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:01.244193    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f512694-9a5d-4d06-9389-856032856ffb" path="/var/lib/kubelet/pods/8f512694-9a5d-4d06-9389-856032856ffb/volumes"
Jul 10 02:17:02.196420 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"1b15242eec14c781e9fd892c23ce4ef82a8a99cffa113a03ad8466956c1067c1"}
Jul 10 02:17:02.196852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"e2fbf10ddd3068ddb08dcbfba8e8c233d821ae284dded381665de04ca8de85c5"}
Jul 10 02:17:02.196852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"f11b34c432100ea30baa25064ef1c20cb8e1e5899bd7a636f9a9db22d20ca02e"}
Jul 10 02:17:02.196852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"bcf7a612a9147738ae3770fd5305afd1926753f76e3880d48a61ce5daf4ad01a"}
Jul 10 02:17:02.196852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196515    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"f725e399a80acd7e9a8ec2c2410d43e12fc7f6f59c8a2beb40d8c3f7adf06988"}
Jul 10 02:17:02.196852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.196537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6a51ba4c-143a-427f-825c-060de64f6c11","Type":"ContainerStarted","Data":"0500f87c82b81cc62229ec8a86f20363ece334b61ba57fb9eb2de2ef1f2fdfc5"}
Jul 10 02:17:02.226055 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:02.226008    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.207620264 podStartE2EDuration="2.225992725s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.372461ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:17:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:17:02.223586224 +0000 UTC m=+147.520168010" watchObservedRunningTime="2026-07-10 02:17:02.225992725 +0000 UTC m=+147.522574499"
Jul 10 02:17:05.558894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:17:05.558860    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:18:00.559081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:00.559043    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:18:00.575207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:00.575178    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:18:01.462980 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:01.462953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 02:18:30.852883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.852842    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"]
Jul 10 02:18:30.855503 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.855480    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:30.858331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.858308    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 02:18:30.859678 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.859664    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 02:18:30.859762 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.859687    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5xqgj\""
Jul 10 02:18:30.866940 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.866918    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"]
Jul 10 02:18:30.920359 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.920321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpcwn\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-kube-api-access-qpcwn\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:30.920522 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:30.920375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.020871 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.020839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpcwn\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-kube-api-access-qpcwn\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.021021 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.020884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.029908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.029888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.030022 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.029964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpcwn\" (UniqueName: \"kubernetes.io/projected/315c34e0-444e-4220-90d9-c1280d075bb3-kube-api-access-qpcwn\") pod \"cert-manager-cainjector-559cdd8c87-9qrmt\" (UID: \"315c34e0-444e-4220-90d9-c1280d075bb3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.172928 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.172828    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"
Jul 10 02:18:31.302575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.302549    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt"]
Jul 10 02:18:31.305350 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:18:31.305321    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod315c34e0_444e_4220_90d9_c1280d075bb3.slice/crio-f1f9f779199834d7c9f5fbbb1d0d85b4646424adf5893fafd76448f6508066f0 WatchSource:0}: Error finding container f1f9f779199834d7c9f5fbbb1d0d85b4646424adf5893fafd76448f6508066f0: Status 404 returned error can't find the container with id f1f9f779199834d7c9f5fbbb1d0d85b4646424adf5893fafd76448f6508066f0
Jul 10 02:18:31.577393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:31.577357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt" event={"ID":"315c34e0-444e-4220-90d9-c1280d075bb3","Type":"ContainerStarted","Data":"f1f9f779199834d7c9f5fbbb1d0d85b4646424adf5893fafd76448f6508066f0"}
Jul 10 02:18:34.595457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:34.595417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt" event={"ID":"315c34e0-444e-4220-90d9-c1280d075bb3","Type":"ContainerStarted","Data":"1e833a4d4e9e87c6d16bee8e917095537a5f95bbd91d9bfc7292e87ff661563f"}
Jul 10 02:18:34.614460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:34.614408    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9qrmt" podStartSLOduration=1.660700582 podStartE2EDuration="4.614390883s" totalImagesPullingTime="2.953690301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:34.612767076 +0000 UTC m=+239.909348901" watchObservedRunningTime="2026-07-10 02:18:34.614390883 +0000 UTC m=+239.910972657"
Jul 10 02:18:39.758601 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.758566    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5dgg"]
Jul 10 02:18:39.760867 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.760850    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.763372 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.763343    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dfmws\""
Jul 10 02:18:39.770666 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.770642    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5dgg"]
Jul 10 02:18:39.801201 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.801175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkg75\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-kube-api-access-gkg75\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.801328 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.801228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.902144 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.902111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkg75\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-kube-api-access-gkg75\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.902305 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.902179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.911804 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.911773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-bound-sa-token\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:39.911949 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:39.911812    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkg75\" (UniqueName: \"kubernetes.io/projected/534efd57-1685-45ed-8adb-90fdbb7991e1-kube-api-access-gkg75\") pod \"cert-manager-55b4d954d5-f5dgg\" (UID: \"534efd57-1685-45ed-8adb-90fdbb7991e1\") " pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:40.070329 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:40.070297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-f5dgg"
Jul 10 02:18:40.204200 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:40.204176    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-f5dgg"]
Jul 10 02:18:40.206380 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:18:40.206351    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod534efd57_1685_45ed_8adb_90fdbb7991e1.slice/crio-dec6d62536da081cb5e1b246abb76654ea856181682f3c540d200d0e21c3a70a WatchSource:0}: Error finding container dec6d62536da081cb5e1b246abb76654ea856181682f3c540d200d0e21c3a70a: Status 404 returned error can't find the container with id dec6d62536da081cb5e1b246abb76654ea856181682f3c540d200d0e21c3a70a
Jul 10 02:18:40.622592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:40.622558    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f5dgg" event={"ID":"534efd57-1685-45ed-8adb-90fdbb7991e1","Type":"ContainerStarted","Data":"cb3568b7a6b42be36576c22283a1a19664751d94d0a8231218b790db9faea094"}
Jul 10 02:18:40.622592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:40.622599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-f5dgg" event={"ID":"534efd57-1685-45ed-8adb-90fdbb7991e1","Type":"ContainerStarted","Data":"dec6d62536da081cb5e1b246abb76654ea856181682f3c540d200d0e21c3a70a"}
Jul 10 02:18:40.641275 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:18:40.641218    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-f5dgg" podStartSLOduration=1.6412020360000001 podStartE2EDuration="1.641202036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:40.639497453 +0000 UTC m=+245.936079229" watchObservedRunningTime="2026-07-10 02:18:40.641202036 +0000 UTC m=+245.937783810"
Jul 10 02:19:01.315368 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.315277    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"]
Jul 10 02:19:01.319750 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.319715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.322458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.322435    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 02:19:01.322619 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.322435    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 02:19:01.322619 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.322591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p8792\""
Jul 10 02:19:01.322983 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.322904    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 02:19:01.323306 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.323286    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 02:19:01.335927 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.335905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"]
Jul 10 02:19:01.382920 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.382886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.382920 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.382918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.383212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.382957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6tjp\" (UniqueName: \"kubernetes.io/projected/3435c26f-6547-4282-bc9f-f0117d64ee06-kube-api-access-j6tjp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.383212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.383040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.383212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.383089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.483817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.483780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.483817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.483816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.484065 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.483858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6tjp\" (UniqueName: \"kubernetes.io/projected/3435c26f-6547-4282-bc9f-f0117d64ee06-kube-api-access-j6tjp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.484065 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.483889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.484065 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.483913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.484314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.484293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.486609 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.486583    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3435c26f-6547-4282-bc9f-f0117d64ee06-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.486723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.486590    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.486824 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.486798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3435c26f-6547-4282-bc9f-f0117d64ee06-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.493395 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.493370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6tjp\" (UniqueName: \"kubernetes.io/projected/3435c26f-6547-4282-bc9f-f0117d64ee06-kube-api-access-j6tjp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-2f4rw\" (UID: \"3435c26f-6547-4282-bc9f-f0117d64ee06\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.632895 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.632810    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:01.786055 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:01.786032    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"]
Jul 10 02:19:01.788626 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:19:01.788602    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3435c26f_6547_4282_bc9f_f0117d64ee06.slice/crio-3c1e1d08c383fc4356662fc89d5975388e34a35b8d9117fbd22198f598868256 WatchSource:0}: Error finding container 3c1e1d08c383fc4356662fc89d5975388e34a35b8d9117fbd22198f598868256: Status 404 returned error can't find the container with id 3c1e1d08c383fc4356662fc89d5975388e34a35b8d9117fbd22198f598868256
Jul 10 02:19:02.721939 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:02.721893    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw" event={"ID":"3435c26f-6547-4282-bc9f-f0117d64ee06","Type":"ContainerStarted","Data":"3c1e1d08c383fc4356662fc89d5975388e34a35b8d9117fbd22198f598868256"}
Jul 10 02:19:05.738530 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:05.738494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw" event={"ID":"3435c26f-6547-4282-bc9f-f0117d64ee06","Type":"ContainerStarted","Data":"72dd9f62054fe9aad103bdef22f789ce62215012a3778a23a8f6188cf03038f3"}
Jul 10 02:19:06.744778 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:06.744717    2579 generic.go:356] "Generic (PLEG): container finished" podID="3435c26f-6547-4282-bc9f-f0117d64ee06" containerID="72dd9f62054fe9aad103bdef22f789ce62215012a3778a23a8f6188cf03038f3" exitCode=0
Jul 10 02:19:06.744778 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:06.744761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw" event={"ID":"3435c26f-6547-4282-bc9f-f0117d64ee06","Type":"ContainerDied","Data":"72dd9f62054fe9aad103bdef22f789ce62215012a3778a23a8f6188cf03038f3"}
Jul 10 02:19:07.751850 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:07.751815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw" event={"ID":"3435c26f-6547-4282-bc9f-f0117d64ee06","Type":"ContainerStarted","Data":"af2dc319d1e4e10ff23911f5c2f7220fff517bb6d8b19e3949fbc9be66b7b61e"}
Jul 10 02:19:07.752235 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:07.751944    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:07.775650 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:07.775596    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw" podStartSLOduration=2.203448078 podStartE2EDuration="6.775575314s" totalImagesPullingTime="3.482508532s" totalInitContainerRuntime="1.089618704s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:07.773677174 +0000 UTC m=+273.070258950" watchObservedRunningTime="2026-07-10 02:19:07.775575314 +0000 UTC m=+273.072157090"
Jul 10 02:19:18.758393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:18.758364    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-2f4rw"
Jul 10 02:19:21.210023 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.209986    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"]
Jul 10 02:19:21.220471 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.220447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.223628 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.223602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 02:19:21.225076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225054    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:19:21.225202 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 02:19:21.225264 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225209    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 02:19:21.225323 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225099    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kdq8g\""
Jul 10 02:19:21.225381 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225120    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 02:19:21.225987 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.225939    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"]
Jul 10 02:19:21.260923 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.260892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cdcdefb7-f388-4521-934e-b3317d09e02f-manager-config\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.260923 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.260921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.261136 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.260940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxmg\" (UniqueName: \"kubernetes.io/projected/cdcdefb7-f388-4521-934e-b3317d09e02f-kube-api-access-6jxmg\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.261136 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.261057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-metrics-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.361864 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.361831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cdcdefb7-f388-4521-934e-b3317d09e02f-manager-config\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.361864 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.361869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.362110 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.361885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxmg\" (UniqueName: \"kubernetes.io/projected/cdcdefb7-f388-4521-934e-b3317d09e02f-kube-api-access-6jxmg\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.362110 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.361931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-metrics-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.362693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.362618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cdcdefb7-f388-4521-934e-b3317d09e02f-manager-config\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.364685 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.364665    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-metrics-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.364798 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.364703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdcdefb7-f388-4521-934e-b3317d09e02f-cert\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.387712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.387691    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxmg\" (UniqueName: \"kubernetes.io/projected/cdcdefb7-f388-4521-934e-b3317d09e02f-kube-api-access-6jxmg\") pod \"lws-controller-manager-865944fd75-mktk9\" (UID: \"cdcdefb7-f388-4521-934e-b3317d09e02f\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.532812 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.532779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:21.687245 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.687215    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"]
Jul 10 02:19:21.689645 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:19:21.689602    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdcdefb7_f388_4521_934e_b3317d09e02f.slice/crio-1a41e8cf0bc1c897f9706752693a9532062746b4b74717844529cfc01601e05e WatchSource:0}: Error finding container 1a41e8cf0bc1c897f9706752693a9532062746b4b74717844529cfc01601e05e: Status 404 returned error can't find the container with id 1a41e8cf0bc1c897f9706752693a9532062746b4b74717844529cfc01601e05e
Jul 10 02:19:21.818214 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:21.818100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9" event={"ID":"cdcdefb7-f388-4521-934e-b3317d09e02f","Type":"ContainerStarted","Data":"1a41e8cf0bc1c897f9706752693a9532062746b4b74717844529cfc01601e05e"}
Jul 10 02:19:22.546603 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.546560    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vprs7"]
Jul 10 02:19:22.550804 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.550774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:22.553930 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.553912    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-dzqcd\""
Jul 10 02:19:22.554368 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.554350    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 02:19:22.562874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.562783    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vprs7"]
Jul 10 02:19:22.577509 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.577476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:22.577646 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.577525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwgl8\" (UniqueName: \"kubernetes.io/projected/624c704a-7936-4aa0-9c76-e5f9080f2209-kube-api-access-cwgl8\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:22.678966 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.678546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:22.678966 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.678595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwgl8\" (UniqueName: \"kubernetes.io/projected/624c704a-7936-4aa0-9c76-e5f9080f2209-kube-api-access-cwgl8\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:22.678966 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:19:22.678697    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 10 02:19:22.678966 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:19:22.678968    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs podName:624c704a-7936-4aa0-9c76-e5f9080f2209 nodeName:}" failed. No retries permitted until 2026-07-10 02:19:23.178842392 +0000 UTC m=+288.475424145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs") pod "model-serving-api-54d89bd5fc-vprs7" (UID: "624c704a-7936-4aa0-9c76-e5f9080f2209") : secret "model-serving-api-tls" not found
Jul 10 02:19:22.721431 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:22.721394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwgl8\" (UniqueName: \"kubernetes.io/projected/624c704a-7936-4aa0-9c76-e5f9080f2209-kube-api-access-cwgl8\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:23.184179 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:23.184116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:23.186946 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:23.186920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/624c704a-7936-4aa0-9c76-e5f9080f2209-tls-certs\") pod \"model-serving-api-54d89bd5fc-vprs7\" (UID: \"624c704a-7936-4aa0-9c76-e5f9080f2209\") " pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:23.470714 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:23.470630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:24.387100 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:24.387072    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vprs7"]
Jul 10 02:19:24.390224 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:19:24.390199    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624c704a_7936_4aa0_9c76_e5f9080f2209.slice/crio-f39a7aba0a93efb64ce4181d945ef3eecb7d3c5176fa378d154cce3525365ffa WatchSource:0}: Error finding container f39a7aba0a93efb64ce4181d945ef3eecb7d3c5176fa378d154cce3525365ffa: Status 404 returned error can't find the container with id f39a7aba0a93efb64ce4181d945ef3eecb7d3c5176fa378d154cce3525365ffa
Jul 10 02:19:24.836908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:24.836864    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7" event={"ID":"624c704a-7936-4aa0-9c76-e5f9080f2209","Type":"ContainerStarted","Data":"f39a7aba0a93efb64ce4181d945ef3eecb7d3c5176fa378d154cce3525365ffa"}
Jul 10 02:19:24.838225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:24.838199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9" event={"ID":"cdcdefb7-f388-4521-934e-b3317d09e02f","Type":"ContainerStarted","Data":"b3f0c2a52cb9e801481a1e3165b995fbdc541303fc1f96a14eee661efd532839"}
Jul 10 02:19:24.838391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:24.838339    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:24.862519 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:24.862463    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9" podStartSLOduration=1.239810065 podStartE2EDuration="3.862443642s" totalImagesPullingTime="2.622633577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:24.861892781 +0000 UTC m=+290.158474555" watchObservedRunningTime="2026-07-10 02:19:24.862443642 +0000 UTC m=+290.159025420"
Jul 10 02:19:26.849447 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:26.849353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7" event={"ID":"624c704a-7936-4aa0-9c76-e5f9080f2209","Type":"ContainerStarted","Data":"d41040d4437e1be363055e89e42fdbaa9c2c88039fd95ce66718185213253f85"}
Jul 10 02:19:26.849856 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:26.849488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:19:26.871093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:26.871049    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7" podStartSLOduration=2.695620767 podStartE2EDuration="4.871032377s" totalImagesPullingTime="2.17541161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:26.869784983 +0000 UTC m=+292.166366781" watchObservedRunningTime="2026-07-10 02:19:26.871032377 +0000 UTC m=+292.167614191"
Jul 10 02:19:29.131031 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.131000    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"]
Jul 10 02:19:29.134825 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.134802    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.137927 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.137906    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 02:19:29.138057 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.138036    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zqphg\""
Jul 10 02:19:29.138344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.138327    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 02:19:29.138838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.138821    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 02:19:29.154318 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.154221    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"]
Jul 10 02:19:29.239345 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.239306    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"]
Jul 10 02:19:29.244969 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.244933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245200 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2txb8\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-kube-api-access-2txb8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245325 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245479 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245465    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245584 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245693 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245678    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245834 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.245977 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.245962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.246093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.246065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.248392 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.248370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.251669 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.251643    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 02:19:29.251813 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.251711    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 02:19:29.252037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.252022    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-55fqv\""
Jul 10 02:19:29.252658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.252640    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 02:19:29.261427 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.261406    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"]
Jul 10 02:19:29.346502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346689 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2txb8\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-kube-api-access-2txb8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346689 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346536    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346689 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.346689 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346919 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346919 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346919 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.346919 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.346883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93ea990-d141-4a68-90cc-f07a1e17228c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347116 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347116 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t54q2\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-kube-api-access-t54q2\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347423 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.347423 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347423 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347766 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347766 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347549    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.347766 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.347655    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.348998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.348977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.349302 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.349284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.360597 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.360573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2txb8\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-kube-api-access-2txb8\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.360680 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.360663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3cbfcff-26cb-4ac3-8f89-245707fab97c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678bjn9c\" (UID: \"e3cbfcff-26cb-4ac3-8f89-245707fab97c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.444898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.444813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:29.448205 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93ea990-d141-4a68-90cc-f07a1e17228c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t54q2\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-kube-api-access-t54q2\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.448556 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.448411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.449101 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.449040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.451503 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.451402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.451791 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.451692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.451791 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.451752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93ea990-d141-4a68-90cc-f07a1e17228c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.452222 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.452194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c93ea990-d141-4a68-90cc-f07a1e17228c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.464646 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.464620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.470297 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.470277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t54q2\" (UniqueName: \"kubernetes.io/projected/c93ea990-d141-4a68-90cc-f07a1e17228c-kube-api-access-t54q2\") pod \"istiod-openshift-gateway-94bb8fbfd-kmf6d\" (UID: \"c93ea990-d141-4a68-90cc-f07a1e17228c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.559481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.559446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:29.597815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.597790    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"]
Jul 10 02:19:29.599908 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:19:29.599882    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cbfcff_26cb_4ac3_8f89_245707fab97c.slice/crio-fa717d68cbc53e70a6f145e7f8667f3c29d9fce6bb4de5221f1e1f33cc2e3446 WatchSource:0}: Error finding container fa717d68cbc53e70a6f145e7f8667f3c29d9fce6bb4de5221f1e1f33cc2e3446: Status 404 returned error can't find the container with id fa717d68cbc53e70a6f145e7f8667f3c29d9fce6bb4de5221f1e1f33cc2e3446
Jul 10 02:19:29.700028 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.699965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"]
Jul 10 02:19:29.702109 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:19:29.702083    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93ea990_d141_4a68_90cc_f07a1e17228c.slice/crio-1d1e4caf95aa7dc9a00e828bb5a98f85181808a77ed8de9542034c1dc08ea5ad WatchSource:0}: Error finding container 1d1e4caf95aa7dc9a00e828bb5a98f85181808a77ed8de9542034c1dc08ea5ad: Status 404 returned error can't find the container with id 1d1e4caf95aa7dc9a00e828bb5a98f85181808a77ed8de9542034c1dc08ea5ad
Jul 10 02:19:29.865851 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.865814    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c" event={"ID":"e3cbfcff-26cb-4ac3-8f89-245707fab97c","Type":"ContainerStarted","Data":"fa717d68cbc53e70a6f145e7f8667f3c29d9fce6bb4de5221f1e1f33cc2e3446"}
Jul 10 02:19:29.867342 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:29.867311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d" event={"ID":"c93ea990-d141-4a68-90cc-f07a1e17228c","Type":"ContainerStarted","Data":"1d1e4caf95aa7dc9a00e828bb5a98f85181808a77ed8de9542034c1dc08ea5ad"}
Jul 10 02:19:33.422948 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.422892    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:19:33.423265 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.422995    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:19:33.423265 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.423040    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:19:33.424093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.424060    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:19:33.424211 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.424129    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:19:33.896832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.896791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c" event={"ID":"e3cbfcff-26cb-4ac3-8f89-245707fab97c","Type":"ContainerStarted","Data":"a8b15df622d5017fd00b06236c5f09980b704df51f4f38284f40868164f98a90"}
Jul 10 02:19:33.898379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.898353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d" event={"ID":"c93ea990-d141-4a68-90cc-f07a1e17228c","Type":"ContainerStarted","Data":"f23ad0e1e5c22322ab7dbd81dcad867ec6a15862f0f17b7810519e3cf636f2e7"}
Jul 10 02:19:33.898523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.898472    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:33.920938 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.920882    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c" podStartSLOduration=1.101022961 podStartE2EDuration="4.92086188s" totalImagesPullingTime="3.819838919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:33.919994146 +0000 UTC m=+299.216575921" watchObservedRunningTime="2026-07-10 02:19:33.92086188 +0000 UTC m=+299.217443655"
Jul 10 02:19:33.943058 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:33.943012    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d" podStartSLOduration=1.223358362 podStartE2EDuration="4.94300054s" totalImagesPullingTime="3.719642178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:33.941453573 +0000 UTC m=+299.238035350" watchObservedRunningTime="2026-07-10 02:19:33.94300054 +0000 UTC m=+299.239582315"
Jul 10 02:19:34.445477 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:34.445439    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:34.450107 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:34.450082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:34.903808 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:34.903774    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:34.904690 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:34.904671    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678bjn9c"
Jul 10 02:19:34.905324 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:34.905305    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-kmf6d"
Jul 10 02:19:35.103401 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:35.103371    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:19:35.103688 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:35.103666    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:19:35.108110 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:35.108088    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 02:19:35.845793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:35.845766    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-865944fd75-mktk9"
Jul 10 02:19:37.858692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:19:37.858662    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vprs7"
Jul 10 02:21:54.875380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.875340    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"]
Jul 10 02:21:54.878153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.878130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.882684 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.882653    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jlhpm\""
Jul 10 02:21:54.898543 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.898519    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"]
Jul 10 02:21:54.999390 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:54.999701 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:54.999668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k9fh\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-kube-api-access-8k9fh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.100894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.100864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.100894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.100895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k9fh\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-kube-api-access-8k9fh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.100917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.100935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.100978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101131 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101424 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101492 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101649 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101809 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.101868 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.101846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.103609 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.103589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.103723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.103701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.111124 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.111098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.111124 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.111120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k9fh\" (UniqueName: \"kubernetes.io/projected/a173f714-4d5d-4d17-bdfb-7e69ef636c13-kube-api-access-8k9fh\") pod \"maas-default-gateway-openshift-default-5896d4dd49-gk5qh\" (UID: \"a173f714-4d5d-4d17-bdfb-7e69ef636c13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.188845 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.188765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:55.330134 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.330112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"]
Jul 10 02:21:55.340347 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.340305    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:21:55.340449 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.340392    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:21:55.340449 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.340434    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:21:55.593978 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.593942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" event={"ID":"a173f714-4d5d-4d17-bdfb-7e69ef636c13","Type":"ContainerStarted","Data":"293496a782613b6a395cac20ba5ff137576eeea0dc0c1ed9a1cce9878ad7a279"}
Jul 10 02:21:55.594209 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.594025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" event={"ID":"a173f714-4d5d-4d17-bdfb-7e69ef636c13","Type":"ContainerStarted","Data":"98fdf65f36bb71284c8447da03097cd1bcfa1350003e04e2ad6dd73bb5fddf46"}
Jul 10 02:21:55.615094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:55.615042    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podStartSLOduration=1.615023126 podStartE2EDuration="1.615023126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:21:55.613281867 +0000 UTC m=+440.909863644" watchObservedRunningTime="2026-07-10 02:21:55.615023126 +0000 UTC m=+440.911604902"
Jul 10 02:21:56.189247 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:56.189210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:56.193559 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:56.193531    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:56.599740 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:56.599699    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:21:56.600642 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:21:56.600623    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:23:31.053368 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.053286    2579 generic.go:356] "Generic (PLEG): container finished" podID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerID="293496a782613b6a395cac20ba5ff137576eeea0dc0c1ed9a1cce9878ad7a279" exitCode=0
Jul 10 02:23:31.053795 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.053363    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" event={"ID":"a173f714-4d5d-4d17-bdfb-7e69ef636c13","Type":"ContainerDied","Data":"293496a782613b6a395cac20ba5ff137576eeea0dc0c1ed9a1cce9878ad7a279"}
Jul 10 02:23:31.054203 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.054189    2579 scope.go:117] "RemoveContainer" containerID="293496a782613b6a395cac20ba5ff137576eeea0dc0c1ed9a1cce9878ad7a279"
Jul 10 02:23:31.055207 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.055178    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:23:31.055260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.055246    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:23:31.055299 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:31.055277    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:23:32.060038 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:32.059999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" event={"ID":"a173f714-4d5d-4d17-bdfb-7e69ef636c13","Type":"ContainerStarted","Data":"5904883cef8765c55fb81ffa585329e7f051ffb7fb3314cc5353d65d362f6645"}
Jul 10 02:23:32.189254 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:32.189222    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:23:32.189484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:32.189411    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gk5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:23:32.189587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:32.189557    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podUID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 10 02:23:33.190103 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:33.190066    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gk5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:23:33.190681 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:33.190649    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podUID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 10 02:23:34.189413 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:34.189376    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gk5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:23:34.189705 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:34.189672    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podUID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 10 02:23:35.189707 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:35.189672    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gk5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:23:35.190812 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:35.190778    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podUID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 10 02:23:36.189639 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:36.189598    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-gk5qh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:23:36.189981 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:36.189946    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh" podUID="a173f714-4d5d-4d17-bdfb-7e69ef636c13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Jul 10 02:23:37.193718 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:37.193689    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:23:37.194196 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:37.193993    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:23:37.194705 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:37.194684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-gk5qh"
Jul 10 02:23:41.565672 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.565643    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"]
Jul 10 02:23:41.568763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.568721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.573127 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.573105    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 02:23:41.573240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.573148    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g7bnd\""
Jul 10 02:23:41.574515 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.574495    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 02:23:41.574606 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.574572    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 10 02:23:41.588666 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.588643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"]
Jul 10 02:23:41.680398 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95tjf\" (UniqueName: \"kubernetes.io/projected/d6042d4a-8779-4b3d-934f-2263a9145ca4-kube-api-access-95tjf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.680398 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.680599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.680599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6042d4a-8779-4b3d-934f-2263a9145ca4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.680599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.680599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.680584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781133 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95tjf\" (UniqueName: \"kubernetes.io/projected/d6042d4a-8779-4b3d-934f-2263a9145ca4-kube-api-access-95tjf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781133 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781178    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6042d4a-8779-4b3d-934f-2263a9145ca4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781399 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781674 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781823 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.781823 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.781758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.783665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.783645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6042d4a-8779-4b3d-934f-2263a9145ca4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.784169 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.784146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6042d4a-8779-4b3d-934f-2263a9145ca4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.794492 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.794474    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95tjf\" (UniqueName: \"kubernetes.io/projected/d6042d4a-8779-4b3d-934f-2263a9145ca4-kube-api-access-95tjf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt\" (UID: \"d6042d4a-8779-4b3d-934f-2263a9145ca4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:41.879181 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:41.879098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:42.023269 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:42.023246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"]
Jul 10 02:23:42.025456 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:23:42.025435    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6042d4a_8779_4b3d_934f_2263a9145ca4.slice/crio-94455f1ed4c8757af124f5e28301c4332b09f434473e461096d19af83bb82bdf WatchSource:0}: Error finding container 94455f1ed4c8757af124f5e28301c4332b09f434473e461096d19af83bb82bdf: Status 404 returned error can't find the container with id 94455f1ed4c8757af124f5e28301c4332b09f434473e461096d19af83bb82bdf
Jul 10 02:23:42.026787 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:42.026770    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:23:42.110075 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:42.110041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt" event={"ID":"d6042d4a-8779-4b3d-934f-2263a9145ca4","Type":"ContainerStarted","Data":"94455f1ed4c8757af124f5e28301c4332b09f434473e461096d19af83bb82bdf"}
Jul 10 02:23:43.118018 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:43.117984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt" event={"ID":"d6042d4a-8779-4b3d-934f-2263a9145ca4","Type":"ContainerStarted","Data":"47f9d58f991743706059017373ea5ab8a5f7515924046e5aff31943a45763bcc"}
Jul 10 02:23:43.148066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:43.148018    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt" podStartSLOduration=1.15453308 podStartE2EDuration="2.148000809s" totalImagesPullingTime="993.467729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:43.146565123 +0000 UTC m=+548.443146898" watchObservedRunningTime="2026-07-10 02:23:43.148000809 +0000 UTC m=+548.444582583"
Jul 10 02:23:51.879265 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:51.879224    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:51.879771 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:51.879288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:51.891824 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:51.891805    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:23:52.179364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:23:52.179271    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt"
Jul 10 02:24:06.366301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.366272    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"]
Jul 10 02:24:06.369045 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.369028    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.371811 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.371790    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 02:24:06.381007 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.380981    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"]
Jul 10 02:24:06.513723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.513909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.513909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.513909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.514071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.514071 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.513989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnhf\" (UniqueName: \"kubernetes.io/projected/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-kube-api-access-lsnhf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.614517 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.614489    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.614757 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.614530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.614757 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.614558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.614988 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.614967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.614988 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.614982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.615143 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.615021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.615143 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.615070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.615143 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.615125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnhf\" (UniqueName: \"kubernetes.io/projected/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-kube-api-access-lsnhf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.615434 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.615297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.617268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.617206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.617675 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.617657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.627314 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.627293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnhf\" (UniqueName: \"kubernetes.io/projected/4fbc297e-c1c3-4afe-8b26-3f98a82c41f5-kube-api-access-lsnhf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pcqhz\" (UID: \"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.679689 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.679654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:06.823856 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:06.823833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"]
Jul 10 02:24:06.825551 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:24:06.825522    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbc297e_c1c3_4afe_8b26_3f98a82c41f5.slice/crio-ee336f7b4f7696e05a66ab49d898912d0c17eaafd2657afaea1f7eb74a85b208 WatchSource:0}: Error finding container ee336f7b4f7696e05a66ab49d898912d0c17eaafd2657afaea1f7eb74a85b208: Status 404 returned error can't find the container with id ee336f7b4f7696e05a66ab49d898912d0c17eaafd2657afaea1f7eb74a85b208
Jul 10 02:24:07.244821 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:07.244719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz" event={"ID":"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5","Type":"ContainerStarted","Data":"3ac3ba5f711a25247d67db1533d2f4e98f95750a456fe1a0d3d4d054602fe20c"}
Jul 10 02:24:07.244821 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:07.244786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz" event={"ID":"4fbc297e-c1c3-4afe-8b26-3f98a82c41f5","Type":"ContainerStarted","Data":"ee336f7b4f7696e05a66ab49d898912d0c17eaafd2657afaea1f7eb74a85b208"}
Jul 10 02:24:07.262410 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:07.262357    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz" podStartSLOduration=1.09851757 podStartE2EDuration="1.262342698s" totalImagesPullingTime="163.825128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:07.261047561 +0000 UTC m=+572.557629337" watchObservedRunningTime="2026-07-10 02:24:07.262342698 +0000 UTC m=+572.558924481"
Jul 10 02:24:16.679789 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:16.679753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:16.680202 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:16.679858    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:16.692324 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:16.692291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:17.305711 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:17.305682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pcqhz"
Jul 10 02:24:35.135859 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:35.135831    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:24:35.136608 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:24:35.136588    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:29:35.167809 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:29:35.167719    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:29:35.170798 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:29:35.170778    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:30:00.142810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.142771    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:30:00.145264 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.145247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"
Jul 10 02:30:00.148274 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.148254    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2pvrv\""
Jul 10 02:30:00.155122 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.155103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"
Jul 10 02:30:00.164138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.164112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:30:00.294265 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.294236    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:30:00.296319 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:30:00.296290    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8b66f0_a0fa_4005_b971_24a801a36453.slice/crio-8da2370f9bc5d2b34ec5358b2ddb10008477a3a4e32ec4d49585bf430c148fb1 WatchSource:0}: Error finding container 8da2370f9bc5d2b34ec5358b2ddb10008477a3a4e32ec4d49585bf430c148fb1: Status 404 returned error can't find the container with id 8da2370f9bc5d2b34ec5358b2ddb10008477a3a4e32ec4d49585bf430c148fb1
Jul 10 02:30:00.298076 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.298055    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:30:00.993017 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:00.992981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" event={"ID":"5a8b66f0-a0fa-4005-b971-24a801a36453","Type":"ContainerStarted","Data":"8da2370f9bc5d2b34ec5358b2ddb10008477a3a4e32ec4d49585bf430c148fb1"}
Jul 10 02:30:03.005470 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:03.005382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" event={"ID":"5a8b66f0-a0fa-4005-b971-24a801a36453","Type":"ContainerStarted","Data":"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"}
Jul 10 02:30:03.023429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:30:03.023386    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" podStartSLOduration=0.606057443 podStartE2EDuration="3.023370719s" totalImagesPullingTime="2.417313276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:30:03.022308725 +0000 UTC m=+928.318890513" watchObservedRunningTime="2026-07-10 02:30:03.023370719 +0000 UTC m=+928.319952495"
Jul 10 02:31:00.011091 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:00.011017    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:31:00.011592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:00.011231    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" podUID="5a8b66f0-a0fa-4005-b971-24a801a36453" containerName="cleanup" containerID="cri-o://6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc" gracePeriod=30
Jul 10 02:31:30.191100 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.191071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"
Jul 10 02:31:30.437162 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.437080    2579 generic.go:356] "Generic (PLEG): container finished" podID="5a8b66f0-a0fa-4005-b971-24a801a36453" containerID="6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc" exitCode=137
Jul 10 02:31:30.437162 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.437118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" event={"ID":"5a8b66f0-a0fa-4005-b971-24a801a36453","Type":"ContainerDied","Data":"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"}
Jul 10 02:31:30.437162 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.437131    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"
Jul 10 02:31:30.437162 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.437155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq" event={"ID":"5a8b66f0-a0fa-4005-b971-24a801a36453","Type":"ContainerDied","Data":"8da2370f9bc5d2b34ec5358b2ddb10008477a3a4e32ec4d49585bf430c148fb1"}
Jul 10 02:31:30.437429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.437176    2579 scope.go:117] "RemoveContainer" containerID="6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"
Jul 10 02:31:30.446448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.446430    2579 scope.go:117] "RemoveContainer" containerID="6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"
Jul 10 02:31:30.446723 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:31:30.446704    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc\": container with ID starting with 6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc not found: ID does not exist" containerID="6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"
Jul 10 02:31:30.446829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.446762    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc"} err="failed to get container status \"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc\": rpc error: code = NotFound desc = could not find container \"6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc\": container with ID starting with 6879c20a1e838a0218b14000b3f6aeb67600e72b6254caa6d0b2c06b197a69bc not found: ID does not exist"
Jul 10 02:31:30.458942 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.458916    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:31:30.462494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:30.462474    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727510-57czq"]
Jul 10 02:31:31.244623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:31:31.244591    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8b66f0-a0fa-4005-b971-24a801a36453" path="/var/lib/kubelet/pods/5a8b66f0-a0fa-4005-b971-24a801a36453/volumes"
Jul 10 02:34:35.200983 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:34:35.200872    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:34:35.204431 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:34:35.203467    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:39:16.419692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.419187    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:16.419692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.419664    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8b66f0-a0fa-4005-b971-24a801a36453" containerName="cleanup"
Jul 10 02:39:16.419692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.419677    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8b66f0-a0fa-4005-b971-24a801a36453" containerName="cleanup"
Jul 10 02:39:16.420191 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.419763    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8b66f0-a0fa-4005-b971-24a801a36453" containerName="cleanup"
Jul 10 02:39:16.422958 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.422938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.425909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.425889    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-32ad1e23-openshift-default-dockercfg-6hvq4\""
Jul 10 02:39:16.452561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.452531    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:16.583623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74lb9\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583780    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.583832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.584067 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.584067 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.583909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.684837 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.684837 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74lb9\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685094 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685295 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.684989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685295 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685295 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685295 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.685683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685664    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.686001 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.685979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.687507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.687488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.687781 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.687760    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.693653 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.693628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.693899 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.693881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74lb9\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9\") pod \"e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") " pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.733877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.733853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:16.877656 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.877624    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:16.878750 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:39:16.878711    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07efe5ec_9d6d_4875_b6a9_4164658f0d86.slice/crio-f33095f1e8ef485c47fbf14f9086b60d72a7a39fb3831bbbbaa1f8cba676b2dc WatchSource:0}: Error finding container f33095f1e8ef485c47fbf14f9086b60d72a7a39fb3831bbbbaa1f8cba676b2dc: Status 404 returned error can't find the container with id f33095f1e8ef485c47fbf14f9086b60d72a7a39fb3831bbbbaa1f8cba676b2dc
Jul 10 02:39:16.880992 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.880960    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:16.881066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.881030    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:16.881066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:16.881060    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:17.760683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:17.760648    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" event={"ID":"07efe5ec-9d6d-4875-b6a9-4164658f0d86","Type":"ContainerStarted","Data":"8a7de31c015f5f21211867ff35b2f3dc45a423051e7f305e5796b4d8a6fc531c"}
Jul 10 02:39:17.761084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:17.760700    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" event={"ID":"07efe5ec-9d6d-4875-b6a9-4164658f0d86","Type":"ContainerStarted","Data":"f33095f1e8ef485c47fbf14f9086b60d72a7a39fb3831bbbbaa1f8cba676b2dc"}
Jul 10 02:39:17.801655 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:17.801595    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" podStartSLOduration=1.801573922 podStartE2EDuration="1.801573922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:17.801330554 +0000 UTC m=+1483.097912357" watchObservedRunningTime="2026-07-10 02:39:17.801573922 +0000 UTC m=+1483.098155699"
Jul 10 02:39:18.734300 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:18.734264    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:18.739005 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:18.738978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:18.765839 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:18.765813    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:18.766579 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:18.766564    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:22.912105 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:22.912067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:22.912902 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:22.912864    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" podUID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" containerName="istio-proxy" containerID="cri-o://8a7de31c015f5f21211867ff35b2f3dc45a423051e7f305e5796b4d8a6fc531c" gracePeriod=30
Jul 10 02:39:28.830602 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:28.830569    2579 generic.go:356] "Generic (PLEG): container finished" podID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" containerID="8a7de31c015f5f21211867ff35b2f3dc45a423051e7f305e5796b4d8a6fc531c" exitCode=0
Jul 10 02:39:28.831087 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:28.830634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" event={"ID":"07efe5ec-9d6d-4875-b6a9-4164658f0d86","Type":"ContainerDied","Data":"8a7de31c015f5f21211867ff35b2f3dc45a423051e7f305e5796b4d8a6fc531c"}
Jul 10 02:39:28.965015 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:28.964961    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:29.008191 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008304    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008370 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008347    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008397    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008443    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008488    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74lb9\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.008575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008511    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:29.008575 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008520    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy\") pod \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\" (UID: \"07efe5ec-9d6d-4875-b6a9-4164658f0d86\") "
Jul 10 02:39:29.009167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:29.009167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008933    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.009167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008955    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.009167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.008947    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:29.009533 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.009508    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:29.009608 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.009576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data" (OuterVolumeSpecName: "istio-data") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:29.010845 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.010817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:39:29.011027 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.011008    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token" (OuterVolumeSpecName: "istio-token") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:29.011250 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.011222    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:29.011335 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.011246    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9" (OuterVolumeSpecName: "kube-api-access-74lb9") pod "07efe5ec-9d6d-4875-b6a9-4164658f0d86" (UID: "07efe5ec-9d6d-4875-b6a9-4164658f0d86"). InnerVolumeSpecName "kube-api-access-74lb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110178    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110203    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110212    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110221    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110229    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110236    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74lb9\" (UniqueName: \"kubernetes.io/projected/07efe5ec-9d6d-4875-b6a9-4164658f0d86-kube-api-access-74lb9\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.110258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.110245    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07efe5ec-9d6d-4875-b6a9-4164658f0d86-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:29.353847 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.353819    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:29.354298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.354246    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" containerName="istio-proxy"
Jul 10 02:39:29.354298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.354266    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" containerName="istio-proxy"
Jul 10 02:39:29.354530 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.354402    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" containerName="istio-proxy"
Jul 10 02:39:29.357422 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.357401    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.360391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.360319    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f26f7768-openshift-default-dockercfg-d9svx\""
Jul 10 02:39:29.371612 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.371589    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:29.413331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413295    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413464 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413464 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413464 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413464 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.413654 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.413595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv4dx\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514048 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv4dx\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514238 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514599 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514717 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514596    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514879 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514961 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.514961 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.514934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.516994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.516971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.517210 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.517191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.522431 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.522403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.522606 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.522589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv4dx\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx\") pod \"e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") " pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.669866 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.669785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:29.806271 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.806239    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:29.808091 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:39:29.808065    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4573222_b3f6_447d_9338_8e9be7d483b4.slice/crio-22f6635272ab31c07b9bccf9625e4040cdf8b90b9bb9a3540c0747c40cd1f363 WatchSource:0}: Error finding container 22f6635272ab31c07b9bccf9625e4040cdf8b90b9bb9a3540c0747c40cd1f363: Status 404 returned error can't find the container with id 22f6635272ab31c07b9bccf9625e4040cdf8b90b9bb9a3540c0747c40cd1f363
Jul 10 02:39:29.810222 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.810192    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:29.810281 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.810254    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:29.810326 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.810281    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:29.837093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.837063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerStarted","Data":"22f6635272ab31c07b9bccf9625e4040cdf8b90b9bb9a3540c0747c40cd1f363"}
Jul 10 02:39:29.838489 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.838469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb" event={"ID":"07efe5ec-9d6d-4875-b6a9-4164658f0d86","Type":"ContainerDied","Data":"f33095f1e8ef485c47fbf14f9086b60d72a7a39fb3831bbbbaa1f8cba676b2dc"}
Jul 10 02:39:29.838489 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.838501    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"
Jul 10 02:39:29.838669 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.838523    2579 scope.go:117] "RemoveContainer" containerID="8a7de31c015f5f21211867ff35b2f3dc45a423051e7f305e5796b4d8a6fc531c"
Jul 10 02:39:29.871284 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.871258    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:29.876991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:29.876969    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-32ad1e23-openshift-default-776489c5df-5bzvb"]
Jul 10 02:39:30.559096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.559061    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:30.562175 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.562151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.565057 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.565030    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:39:30.565057 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.565044    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f26f7768-serving-cert\""
Jul 10 02:39:30.565264 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.565072    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:39:30.566267 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.566247    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:39:30.572671 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.572651    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:30.623815 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.623783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l92gq\" (UniqueName: \"kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.623998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.623884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.623998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.623921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.724611 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.724584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l92gq\" (UniqueName: \"kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.724818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.724635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.724818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.724658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.725226 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.725201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.727150 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.727129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.741566 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.741534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l92gq\" (UniqueName: \"kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq\") pod \"maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:30.844717 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.844601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerStarted","Data":"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"}
Jul 10 02:39:30.868519 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.868467    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" podStartSLOduration=1.868450296 podStartE2EDuration="1.868450296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:30.86570057 +0000 UTC m=+1496.162282347" watchObservedRunningTime="2026-07-10 02:39:30.868450296 +0000 UTC m=+1496.165032071"
Jul 10 02:39:30.875218 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:30.875197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:31.014260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.014228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:31.016111 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:39:31.016084    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda093b40_4c20_40f4_aae5_c084e30cb101.slice/crio-9cc849a586d0c24c20010c2167e1ae9945e5d1ead234f1be17eb370f0e810693 WatchSource:0}: Error finding container 9cc849a586d0c24c20010c2167e1ae9945e5d1ead234f1be17eb370f0e810693: Status 404 returned error can't find the container with id 9cc849a586d0c24c20010c2167e1ae9945e5d1ead234f1be17eb370f0e810693
Jul 10 02:39:31.017515 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.017497    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:39:31.244861 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.244785    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07efe5ec-9d6d-4875-b6a9-4164658f0d86" path="/var/lib/kubelet/pods/07efe5ec-9d6d-4875-b6a9-4164658f0d86/volumes"
Jul 10 02:39:31.670309 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.670217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:31.677984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.677955    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:31.855119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.854788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" event={"ID":"da093b40-4c20-40f4-aae5-c084e30cb101","Type":"ContainerStarted","Data":"9cc849a586d0c24c20010c2167e1ae9945e5d1ead234f1be17eb370f0e810693"}
Jul 10 02:39:31.855595 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.855336    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:31.856425 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:31.856403    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:32.863037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:32.862997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" event={"ID":"da093b40-4c20-40f4-aae5-c084e30cb101","Type":"ContainerStarted","Data":"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"}
Jul 10 02:39:32.863496 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:32.863275    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:32.885421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:32.885363    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" podStartSLOduration=1.423718384 podStartE2EDuration="2.885347389s" totalImagesPullingTime="1.461629005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:32.883764339 +0000 UTC m=+1498.180346117" watchObservedRunningTime="2026-07-10 02:39:32.885347389 +0000 UTC m=+1498.181929164"
Jul 10 02:39:35.235991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:35.235965    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:39:35.238320 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:35.238299    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:39:36.036683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.036642    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:36.036942 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.036892    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" podUID="da093b40-4c20-40f4-aae5-c084e30cb101" containerName="maas-api" containerID="cri-o://d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba" gracePeriod=30
Jul 10 02:39:36.338395 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.338364    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:36.381356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.381327    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca\") pod \"da093b40-4c20-40f4-aae5-c084e30cb101\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") "
Jul 10 02:39:36.381558 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.381385    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l92gq\" (UniqueName: \"kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq\") pod \"da093b40-4c20-40f4-aae5-c084e30cb101\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") "
Jul 10 02:39:36.381558 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.381436    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls\") pod \"da093b40-4c20-40f4-aae5-c084e30cb101\" (UID: \"da093b40-4c20-40f4-aae5-c084e30cb101\") "
Jul 10 02:39:36.381781 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.381758    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da093b40-4c20-40f4-aae5-c084e30cb101" (UID: "da093b40-4c20-40f4-aae5-c084e30cb101"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:36.383759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.383707    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq" (OuterVolumeSpecName: "kube-api-access-l92gq") pod "da093b40-4c20-40f4-aae5-c084e30cb101" (UID: "da093b40-4c20-40f4-aae5-c084e30cb101"). InnerVolumeSpecName "kube-api-access-l92gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:36.383759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.383712    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da093b40-4c20-40f4-aae5-c084e30cb101" (UID: "da093b40-4c20-40f4-aae5-c084e30cb101"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:39:36.483243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.483174    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da093b40-4c20-40f4-aae5-c084e30cb101-maas-api-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:36.483243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.483204    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da093b40-4c20-40f4-aae5-c084e30cb101-openshift-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:36.483243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.483218    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l92gq\" (UniqueName: \"kubernetes.io/projected/da093b40-4c20-40f4-aae5-c084e30cb101-kube-api-access-l92gq\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:36.890456 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.890410    2579 generic.go:356] "Generic (PLEG): container finished" podID="da093b40-4c20-40f4-aae5-c084e30cb101" containerID="d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba" exitCode=0
Jul 10 02:39:36.890660 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.890513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" event={"ID":"da093b40-4c20-40f4-aae5-c084e30cb101","Type":"ContainerDied","Data":"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"}
Jul 10 02:39:36.890660 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.890565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc" event={"ID":"da093b40-4c20-40f4-aae5-c084e30cb101","Type":"ContainerDied","Data":"9cc849a586d0c24c20010c2167e1ae9945e5d1ead234f1be17eb370f0e810693"}
Jul 10 02:39:36.890660 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.890596    2579 scope.go:117] "RemoveContainer" containerID="d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"
Jul 10 02:39:36.890883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.890787    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"
Jul 10 02:39:36.918876 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.918854    2579 scope.go:117] "RemoveContainer" containerID="d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"
Jul 10 02:39:36.919205 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:39:36.919187    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba\": container with ID starting with d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba not found: ID does not exist" containerID="d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"
Jul 10 02:39:36.919277 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.919219    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba"} err="failed to get container status \"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba\": rpc error: code = NotFound desc = could not find container \"d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba\": container with ID starting with d3c5e5d2c11b63b8253d137d60af8bba229d0d2c81ba1c2a1dd02ee3044633ba not found: ID does not exist"
Jul 10 02:39:36.928151 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.928124    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:36.931911 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:36.931885    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f26f7768-7474dd6ddb-v4wpc"]
Jul 10 02:39:37.246038 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:37.245960    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da093b40-4c20-40f4-aae5-c084e30cb101" path="/var/lib/kubelet/pods/da093b40-4c20-40f4-aae5-c084e30cb101/volumes"
Jul 10 02:39:40.916682 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.916597    2579 generic.go:356] "Generic (PLEG): container finished" podID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerID="825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9" exitCode=0
Jul 10 02:39:40.917111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.916677    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerDied","Data":"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"}
Jul 10 02:39:40.917162 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.917120    2579 scope.go:117] "RemoveContainer" containerID="825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"
Jul 10 02:39:40.918237 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.918198    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:40.918339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.918286    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:40.918402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:40.918341    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:41.831088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:41.831052    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:41.923334 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:41.923297    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerStarted","Data":"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"}
Jul 10 02:39:41.923759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:41.923379    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy" containerID="cri-o://90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af" gracePeriod=30
Jul 10 02:39:46.855644 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:46.855606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:47.180000 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.179973    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:47.303779 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.303744    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv4dx\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.303942 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.303801    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.303942 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.303855    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.303942 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.303917    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.303954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304032    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304084    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs\") pod \"f4573222-b3f6-447d-9338-8e9be7d483b4\" (UID: \"f4573222-b3f6-447d-9338-8e9be7d483b4\") "
Jul 10 02:39:47.304340 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304205    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:47.304457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:47.304536 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304516    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.304593 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304545    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.304682 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304654    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:47.304777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304757    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:47.305022 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.304989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:47.306240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.306216    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:47.306606 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.306587    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:39:47.306905 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.306881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx" (OuterVolumeSpecName: "kube-api-access-hv4dx") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "kube-api-access-hv4dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:47.306984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.306942    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4573222-b3f6-447d-9338-8e9be7d483b4" (UID: "f4573222-b3f6-447d-9338-8e9be7d483b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:47.405817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405778    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.405817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405807    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.405817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405816    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4573222-b3f6-447d-9338-8e9be7d483b4-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.405817 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405824    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.406164 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405832    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.406164 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405840    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv4dx\" (UniqueName: \"kubernetes.io/projected/f4573222-b3f6-447d-9338-8e9be7d483b4-kube-api-access-hv4dx\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.406164 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.405849    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4573222-b3f6-447d-9338-8e9be7d483b4-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:47.955001 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.954969    2579 generic.go:356] "Generic (PLEG): container finished" podID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerID="90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af" exitCode=0
Jul 10 02:39:47.955417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.955026    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"
Jul 10 02:39:47.955417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.955028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerDied","Data":"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"}
Jul 10 02:39:47.955417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.955150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8" event={"ID":"f4573222-b3f6-447d-9338-8e9be7d483b4","Type":"ContainerDied","Data":"22f6635272ab31c07b9bccf9625e4040cdf8b90b9bb9a3540c0747c40cd1f363"}
Jul 10 02:39:47.955417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.955179    2579 scope.go:117] "RemoveContainer" containerID="90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"
Jul 10 02:39:47.964900 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.964884    2579 scope.go:117] "RemoveContainer" containerID="825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"
Jul 10 02:39:47.973603 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.973583    2579 scope.go:117] "RemoveContainer" containerID="90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"
Jul 10 02:39:47.973892 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:39:47.973870    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af\": container with ID starting with 90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af not found: ID does not exist" containerID="90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"
Jul 10 02:39:47.973946 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.973906    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af"} err="failed to get container status \"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af\": rpc error: code = NotFound desc = could not find container \"90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af\": container with ID starting with 90a884bbc14a14225b02db20b42a635c261c321f9d56b852c11979aff77691af not found: ID does not exist"
Jul 10 02:39:47.973946 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.973928    2579 scope.go:117] "RemoveContainer" containerID="825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"
Jul 10 02:39:47.974185 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:39:47.974166    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9\": container with ID starting with 825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9 not found: ID does not exist" containerID="825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"
Jul 10 02:39:47.974227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.974193    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9"} err="failed to get container status \"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9\": rpc error: code = NotFound desc = could not find container \"825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9\": container with ID starting with 825937f229694b57b3f3837d0faa0253ff5e76ef9d784728f16771b900108fb9 not found: ID does not exist"
Jul 10 02:39:47.977645 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.977623    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:47.981578 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:47.981556    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f26f7768-openshift-default-77c47d5d6c-dq2q8"]
Jul 10 02:39:48.574436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574394    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:39:48.574874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574840    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.574874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574853    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.574959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574878    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.574959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574887    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.574959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574906    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da093b40-4c20-40f4-aae5-c084e30cb101" containerName="maas-api"
Jul 10 02:39:48.574959 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574917    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da093b40-4c20-40f4-aae5-c084e30cb101" containerName="maas-api"
Jul 10 02:39:48.575077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574984    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.575077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.574991    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="da093b40-4c20-40f4-aae5-c084e30cb101" containerName="maas-api"
Jul 10 02:39:48.575077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.575000    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" containerName="istio-proxy"
Jul 10 02:39:48.579507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.579491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.582436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.582414    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-625bac5e-openshift-default-dockercfg-zrqsf\""
Jul 10 02:39:48.594748 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.594707    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:39:48.717803 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.717773    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.717803 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.717807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.717869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.717896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.717967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718012 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.718001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkhgz\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718145 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.718077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718145 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.718112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.718145 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.718137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819148 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkhgz\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819762 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819844    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.819894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.820005 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.819933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.820153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.820129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.821840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.821818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.822125 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.822107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.828777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.828707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.828955 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.828941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkhgz\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz\") pod \"e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") " pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:48.889962 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:48.889942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:49.028484 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.028453    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:39:49.029883 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:39:49.029859    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd743b4_225c_47ab_80f8_c9f04491d31e.slice/crio-d63ae38ffd14738ef97ad0863c5a56cd86c893ed620427600b284b0edf5e7d85 WatchSource:0}: Error finding container d63ae38ffd14738ef97ad0863c5a56cd86c893ed620427600b284b0edf5e7d85: Status 404 returned error can't find the container with id d63ae38ffd14738ef97ad0863c5a56cd86c893ed620427600b284b0edf5e7d85
Jul 10 02:39:49.032097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.032058    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:49.032171 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.032140    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:49.032214 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.032187    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:39:49.247104 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.247067    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4573222-b3f6-447d-9338-8e9be7d483b4" path="/var/lib/kubelet/pods/f4573222-b3f6-447d-9338-8e9be7d483b4/volumes"
Jul 10 02:39:49.854786 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.854754    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:49.859578 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.859558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:49.862323 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.862295    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-625bac5e-serving-cert\""
Jul 10 02:39:49.862483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.862421    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:39:49.862483 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.862431    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:39:49.863541 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.863526    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:39:49.869352 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.869327    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:49.970455 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.970424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" event={"ID":"acd743b4-225c-47ab-80f8-c9f04491d31e","Type":"ContainerStarted","Data":"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"}
Jul 10 02:39:49.970640 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.970491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" event={"ID":"acd743b4-225c-47ab-80f8-c9f04491d31e","Type":"ContainerStarted","Data":"d63ae38ffd14738ef97ad0863c5a56cd86c893ed620427600b284b0edf5e7d85"}
Jul 10 02:39:49.992949 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:49.992897    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" podStartSLOduration=1.992879196 podStartE2EDuration="1.992879196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:49.990967709 +0000 UTC m=+1515.287549497" watchObservedRunningTime="2026-07-10 02:39:49.992879196 +0000 UTC m=+1515.289460971"
Jul 10 02:39:50.030509 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.030470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.030941 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.030589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l5cx\" (UniqueName: \"kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.030941 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.030717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.132408 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.132294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l5cx\" (UniqueName: \"kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.132568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.132470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.132568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.132517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.133240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.133212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.135056 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.135038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.145504 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.145481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l5cx\" (UniqueName: \"kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx\") pod \"maas-api-e2e-derive-625bac5e-7b569b795d-kq99h\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.172497 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.172474    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.307543 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.307517    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:50.309414 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:39:50.309390    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode01c6c52_f343_4b7c_be3b_20598b818b14.slice/crio-1f677f6984a158d8536312c926ba58b269145315ee638bfd41810c51da95b5ac WatchSource:0}: Error finding container 1f677f6984a158d8536312c926ba58b269145315ee638bfd41810c51da95b5ac: Status 404 returned error can't find the container with id 1f677f6984a158d8536312c926ba58b269145315ee638bfd41810c51da95b5ac
Jul 10 02:39:50.890978 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.890892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:50.895765 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.895714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:50.976158 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.976125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" event={"ID":"e01c6c52-f343-4b7c-be3b-20598b818b14","Type":"ContainerStarted","Data":"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"}
Jul 10 02:39:50.976363 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.976172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" event={"ID":"e01c6c52-f343-4b7c-be3b-20598b818b14","Type":"ContainerStarted","Data":"1f677f6984a158d8536312c926ba58b269145315ee638bfd41810c51da95b5ac"}
Jul 10 02:39:50.976363 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.976200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:50.976530 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.976509    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:50.977353 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.977333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:50.994087 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:50.994040    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" podStartSLOduration=1.6756736920000002 podStartE2EDuration="1.994024812s" totalImagesPullingTime="318.35112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:50.991513941 +0000 UTC m=+1516.288095718" watchObservedRunningTime="2026-07-10 02:39:50.994024812 +0000 UTC m=+1516.290606614"
Jul 10 02:39:54.333502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.333458    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:39:54.334092 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.333799    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" podUID="acd743b4-225c-47ab-80f8-c9f04491d31e" containerName="istio-proxy" containerID="cri-o://bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540" gracePeriod=30
Jul 10 02:39:54.515754 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.515133    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:54.515754 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.515477    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" podUID="e01c6c52-f343-4b7c-be3b-20598b818b14" containerName="maas-api" containerID="cri-o://d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149" gracePeriod=30
Jul 10 02:39:54.775539 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.775515    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:54.782755 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.782705    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l5cx\" (UniqueName: \"kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx\") pod \"e01c6c52-f343-4b7c-be3b-20598b818b14\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") "
Jul 10 02:39:54.782887 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.782779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls\") pod \"e01c6c52-f343-4b7c-be3b-20598b818b14\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") "
Jul 10 02:39:54.782887 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.782818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca\") pod \"e01c6c52-f343-4b7c-be3b-20598b818b14\" (UID: \"e01c6c52-f343-4b7c-be3b-20598b818b14\") "
Jul 10 02:39:54.783212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.783187    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e01c6c52-f343-4b7c-be3b-20598b818b14" (UID: "e01c6c52-f343-4b7c-be3b-20598b818b14"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:54.812699 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.808721    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e01c6c52-f343-4b7c-be3b-20598b818b14" (UID: "e01c6c52-f343-4b7c-be3b-20598b818b14"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:39:54.816763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.815925    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx" (OuterVolumeSpecName: "kube-api-access-5l5cx") pod "e01c6c52-f343-4b7c-be3b-20598b818b14" (UID: "e01c6c52-f343-4b7c-be3b-20598b818b14"). InnerVolumeSpecName "kube-api-access-5l5cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:54.883554 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.883523    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e01c6c52-f343-4b7c-be3b-20598b818b14-openshift-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:54.883554 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.883551    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l5cx\" (UniqueName: \"kubernetes.io/projected/e01c6c52-f343-4b7c-be3b-20598b818b14-kube-api-access-5l5cx\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:54.883765 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:54.883563    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e01c6c52-f343-4b7c-be3b-20598b818b14-maas-api-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:55.001097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.000996    2579 generic.go:356] "Generic (PLEG): container finished" podID="e01c6c52-f343-4b7c-be3b-20598b818b14" containerID="d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149" exitCode=0
Jul 10 02:39:55.001097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.001083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" event={"ID":"e01c6c52-f343-4b7c-be3b-20598b818b14","Type":"ContainerDied","Data":"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"}
Jul 10 02:39:55.001330 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.001133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h" event={"ID":"e01c6c52-f343-4b7c-be3b-20598b818b14","Type":"ContainerDied","Data":"1f677f6984a158d8536312c926ba58b269145315ee638bfd41810c51da95b5ac"}
Jul 10 02:39:55.001330 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.001159    2579 scope.go:117] "RemoveContainer" containerID="d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"
Jul 10 02:39:55.001330 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.001315    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"
Jul 10 02:39:55.021155 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.021136    2579 scope.go:117] "RemoveContainer" containerID="d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"
Jul 10 02:39:55.021454 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:39:55.021432    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149\": container with ID starting with d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149 not found: ID does not exist" containerID="d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"
Jul 10 02:39:55.021501 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.021465    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149"} err="failed to get container status \"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149\": rpc error: code = NotFound desc = could not find container \"d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149\": container with ID starting with d92c893fb6c81d50cdbce30af1f8fcb3347040e80c3aee78d8fb298e81dcd149 not found: ID does not exist"
Jul 10 02:39:55.029459 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.029438    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:55.033325 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.033303    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-625bac5e-7b569b795d-kq99h"]
Jul 10 02:39:55.246281 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:55.246251    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e01c6c52-f343-4b7c-be3b-20598b818b14" path="/var/lib/kubelet/pods/e01c6c52-f343-4b7c-be3b-20598b818b14/volumes"
Jul 10 02:39:59.597591 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.597527    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:39:59.639378 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.639341    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.639646 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.639629    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.639794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.639776    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.639948 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.639932    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.640109 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.639788    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:59.640184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640075    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:59.640184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640088    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.640184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640169    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.640349 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640249    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkhgz\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.640406 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.640469 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.640433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data\") pod \"acd743b4-225c-47ab-80f8-c9f04491d31e\" (UID: \"acd743b4-225c-47ab-80f8-c9f04491d31e\") "
Jul 10 02:39:59.641156 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.641071    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.641776 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.641748    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd743b4-225c-47ab-80f8-c9f04491d31e-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.644671 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.642359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:59.644671 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.643542    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data" (OuterVolumeSpecName: "istio-data") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:59.644671 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.643629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz" (OuterVolumeSpecName: "kube-api-access-zkhgz") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "kube-api-access-zkhgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:59.645018 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.644964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:59.645350 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.645318    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:39:59.647106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.647079    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token" (OuterVolumeSpecName: "istio-token") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:59.647395 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.647360    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acd743b4-225c-47ab-80f8-c9f04491d31e" (UID: "acd743b4-225c-47ab-80f8-c9f04491d31e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:39:59.743233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743197    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743226    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743238    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743251    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743266    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743277    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd743b4-225c-47ab-80f8-c9f04491d31e-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:39:59.743466 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:39:59.743287    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkhgz\" (UniqueName: \"kubernetes.io/projected/acd743b4-225c-47ab-80f8-c9f04491d31e-kube-api-access-zkhgz\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:40:00.032595 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.032559    2579 generic.go:356] "Generic (PLEG): container finished" podID="acd743b4-225c-47ab-80f8-c9f04491d31e" containerID="bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540" exitCode=0
Jul 10 02:40:00.032794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.032627    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"
Jul 10 02:40:00.032794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.032660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" event={"ID":"acd743b4-225c-47ab-80f8-c9f04491d31e","Type":"ContainerDied","Data":"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"}
Jul 10 02:40:00.032794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.032763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9" event={"ID":"acd743b4-225c-47ab-80f8-c9f04491d31e","Type":"ContainerDied","Data":"d63ae38ffd14738ef97ad0863c5a56cd86c893ed620427600b284b0edf5e7d85"}
Jul 10 02:40:00.032794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.032791    2579 scope.go:117] "RemoveContainer" containerID="bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"
Jul 10 02:40:00.043720 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.043703    2579 scope.go:117] "RemoveContainer" containerID="bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"
Jul 10 02:40:00.044085 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:40:00.044064    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540\": container with ID starting with bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540 not found: ID does not exist" containerID="bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"
Jul 10 02:40:00.044194 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.044095    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540"} err="failed to get container status \"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540\": rpc error: code = NotFound desc = could not find container \"bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540\": container with ID starting with bb102d158818ae456397414e4b37f58dfefffcc638615dbdee011fd3ef5c8540 not found: ID does not exist"
Jul 10 02:40:00.054240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.054215    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:40:00.057601 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:00.057583    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-625bac5e-openshift-default-655df7575f-zkqw9"]
Jul 10 02:40:01.245172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:40:01.245136    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd743b4-225c-47ab-80f8-c9f04491d31e" path="/var/lib/kubelet/pods/acd743b4-225c-47ab-80f8-c9f04491d31e/volumes"
Jul 10 02:41:37.715500 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.715460    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:41:37.716105 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716086    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd743b4-225c-47ab-80f8-c9f04491d31e" containerName="istio-proxy"
Jul 10 02:41:37.716153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716109    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd743b4-225c-47ab-80f8-c9f04491d31e" containerName="istio-proxy"
Jul 10 02:41:37.716153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716142    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e01c6c52-f343-4b7c-be3b-20598b818b14" containerName="maas-api"
Jul 10 02:41:37.716153 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716151    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e01c6c52-f343-4b7c-be3b-20598b818b14" containerName="maas-api"
Jul 10 02:41:37.716252 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716243    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e01c6c52-f343-4b7c-be3b-20598b818b14" containerName="maas-api"
Jul 10 02:41:37.716287 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.716263    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd743b4-225c-47ab-80f8-c9f04491d31e" containerName="istio-proxy"
Jul 10 02:41:37.719505 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.719487    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.722205 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.722185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-57470f4b-openshift-default-dockercfg-59sx5\""
Jul 10 02:41:37.741620 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.741595    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:41:37.887072 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887253 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887253 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887221    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc8lm\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887488 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887488 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.887554 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.887490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988230 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988230 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc8lm\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988230 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988250    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988862 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988663    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988862 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988750    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988945 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.988983 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.988963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.989033 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.989018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.991157 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.991136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.991470 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.991449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.997082 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.997062    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:37.997227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:37.997206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc8lm\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm\") pod \"e2e-mt-57470f4b-openshift-default-7975666d75-tllvh\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") " pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:38.029317 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.029285    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:38.171832 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.171806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:41:38.173684 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:41:38.173655    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aca65e2_83d4_4806_987e_25c3c4234161.slice/crio-e2bd6ddde278902a22e9d320a040570f39c2765a4361346471eb187475f199a6 WatchSource:0}: Error finding container e2bd6ddde278902a22e9d320a040570f39c2765a4361346471eb187475f199a6: Status 404 returned error can't find the container with id e2bd6ddde278902a22e9d320a040570f39c2765a4361346471eb187475f199a6
Jul 10 02:41:38.175838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.175800    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:38.175925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.175870    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:38.175925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.175903    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:38.520624 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.520587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerStarted","Data":"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"}
Jul 10 02:41:38.520826 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.520638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerStarted","Data":"e2bd6ddde278902a22e9d320a040570f39c2765a4361346471eb187475f199a6"}
Jul 10 02:41:38.541838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:38.541785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podStartSLOduration=1.541765882 podStartE2EDuration="1.541765882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:38.540050337 +0000 UTC m=+1623.836632113" watchObservedRunningTime="2026-07-10 02:41:38.541765882 +0000 UTC m=+1623.838347656"
Jul 10 02:41:39.029956 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:39.029922    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:39.035234 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:39.035197    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:39.525356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:39.525330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:39.526253 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:39.526238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:54.527072 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.527036    2579 patch_prober.go:28] interesting pod/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 02:41:54.527491 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.527100    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 02:41:54.603925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.603889    2579 generic.go:356] "Generic (PLEG): container finished" podID="6aca65e2-83d4-4806-987e-25c3c4234161" containerID="b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621" exitCode=0
Jul 10 02:41:54.604115 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.603935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerDied","Data":"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"}
Jul 10 02:41:54.604406 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.604388    2579 scope.go:117] "RemoveContainer" containerID="b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"
Jul 10 02:41:54.605542 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.605510    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:54.605613 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.605595    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:54.605662 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:54.605648    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:41:55.613445 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:55.613404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerStarted","Data":"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"}
Jul 10 02:41:56.029954 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:56.029916    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:41:56.030242 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:56.030217    2579 patch_prober.go:28] interesting pod/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 02:41:56.030298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:56.030277    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 02:41:57.030560 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:57.030523    2579 patch_prober.go:28] interesting pod/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 02:41:57.031000 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:57.030580    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 02:41:58.029702 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:58.029666    2579 patch_prober.go:28] interesting pod/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 02:41:58.030015 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:58.029807    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 02:41:59.030401 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:59.030365    2579 patch_prober.go:28] interesting pod/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 10 02:41:59.030833 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:59.030420    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 10 02:41:59.759509 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:59.759478    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:41:59.759722 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:41:59.759696    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy" containerID="cri-o://f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b" gracePeriod=30
Jul 10 02:42:05.011371 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.011345    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:42:05.170345 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qc8lm\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170345 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170306    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170357    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170397    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170521    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170550    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170598    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs\") pod \"6aca65e2-83d4-4806-987e-25c3c4234161\" (UID: \"6aca65e2-83d4-4806-987e-25c3c4234161\") "
Jul 10 02:42:05.170852 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170759    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:05.171138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170938    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:05.171138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.170959    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:05.171138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.171053    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.171138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.171069    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6aca65e2-83d4-4806-987e-25c3c4234161-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.171138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.171083    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.171432 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.171296    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:05.171432 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.171399    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data" (OuterVolumeSpecName: "istio-data") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:05.172723 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.172693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm" (OuterVolumeSpecName: "kube-api-access-qc8lm") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "kube-api-access-qc8lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:05.173186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.173166    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:05.173186 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.173171    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token" (OuterVolumeSpecName: "istio-token") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:05.173289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.173199    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6aca65e2-83d4-4806-987e-25c3c4234161" (UID: "6aca65e2-83d4-4806-987e-25c3c4234161"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:42:05.272482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272450    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.272482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272478    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6aca65e2-83d4-4806-987e-25c3c4234161-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.272482 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272487    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.272696 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272496    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc8lm\" (UniqueName: \"kubernetes.io/projected/6aca65e2-83d4-4806-987e-25c3c4234161-kube-api-access-qc8lm\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.272696 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272506    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.272696 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.272515    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6aca65e2-83d4-4806-987e-25c3c4234161-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:42:05.661941 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.661907    2579 generic.go:356] "Generic (PLEG): container finished" podID="6aca65e2-83d4-4806-987e-25c3c4234161" containerID="f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b" exitCode=0
Jul 10 02:42:05.662126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.661957    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"
Jul 10 02:42:05.662126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.661957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerDied","Data":"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"}
Jul 10 02:42:05.662126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.662012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh" event={"ID":"6aca65e2-83d4-4806-987e-25c3c4234161","Type":"ContainerDied","Data":"e2bd6ddde278902a22e9d320a040570f39c2765a4361346471eb187475f199a6"}
Jul 10 02:42:05.662126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.662035    2579 scope.go:117] "RemoveContainer" containerID="f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"
Jul 10 02:42:05.671321 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.671301    2579 scope.go:117] "RemoveContainer" containerID="b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"
Jul 10 02:42:05.680658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.680621    2579 scope.go:117] "RemoveContainer" containerID="f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"
Jul 10 02:42:05.681024 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:42:05.681001    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b\": container with ID starting with f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b not found: ID does not exist" containerID="f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"
Jul 10 02:42:05.681123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.681036    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b"} err="failed to get container status \"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b\": rpc error: code = NotFound desc = could not find container \"f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b\": container with ID starting with f80582bd5a984448adb31a13ebf823b701f11eb24bdc8562b39227f64206ea3b not found: ID does not exist"
Jul 10 02:42:05.681123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.681058    2579 scope.go:117] "RemoveContainer" containerID="b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"
Jul 10 02:42:05.681228 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.681217    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:42:05.681356 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:42:05.681335    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621\": container with ID starting with b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621 not found: ID does not exist" containerID="b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"
Jul 10 02:42:05.681392 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.681363    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621"} err="failed to get container status \"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621\": rpc error: code = NotFound desc = could not find container \"b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621\": container with ID starting with b218e3af5d23dd1d5136e20429043ea4dfcfb4b92bb58a289fa37e17b6a43621 not found: ID does not exist"
Jul 10 02:42:05.683048 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:05.683029    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-57470f4b-openshift-default-7975666d75-tllvh"]
Jul 10 02:42:07.244755 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:07.244695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" path="/var/lib/kubelet/pods/6aca65e2-83d4-4806-987e-25c3c4234161/volumes"
Jul 10 02:42:19.706688 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.706649    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:42:19.707283 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.707095    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:42:19.707283 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.707108    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:42:19.707283 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.707202    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:42:19.707283 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.707212    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:42:19.711669 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.711647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.714234 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.714209    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ad6c59d4-openshift-default-dockercfg-8prvs\""
Jul 10 02:42:19.723773 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.723035    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:42:19.814010 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.813974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814194 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814194 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814305 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn8qt\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814351 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814450 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814532 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.814590 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.814549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.915926 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.915887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916102 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.915945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916102 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.915990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916102 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916288 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916288 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn8qt\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916288 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916288 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916288 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916250    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.916825 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.916804    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.917079 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.917057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.918868 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.918845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.918991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.918942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.924185 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.924166    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:19.924312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:19.924293    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn8qt\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt\") pod \"e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") " pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:20.028367 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.028340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:20.170683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.170659    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:42:20.172807 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:42:20.172776    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93504187_18ea_4991_b85b_e1c6b59c0398.slice/crio-d257784421bb91ae2d3431832841fb5df3ba3d8f66988ed5c58d3b1eef66912d WatchSource:0}: Error finding container d257784421bb91ae2d3431832841fb5df3ba3d8f66988ed5c58d3b1eef66912d: Status 404 returned error can't find the container with id d257784421bb91ae2d3431832841fb5df3ba3d8f66988ed5c58d3b1eef66912d
Jul 10 02:42:20.175139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.175094    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:20.175230 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.175197    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:20.175289 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.175270    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:20.740491 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.740454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerStarted","Data":"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"}
Jul 10 02:42:20.740921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.740517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerStarted","Data":"d257784421bb91ae2d3431832841fb5df3ba3d8f66988ed5c58d3b1eef66912d"}
Jul 10 02:42:20.763011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:20.762965    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podStartSLOduration=1.762948965 podStartE2EDuration="1.762948965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:20.760827773 +0000 UTC m=+1666.057409548" watchObservedRunningTime="2026-07-10 02:42:20.762948965 +0000 UTC m=+1666.059530740"
Jul 10 02:42:21.029472 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:21.029436    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:21.034883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:21.034860    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:21.747264 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:21.747231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:21.748362 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:21.748343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:37.094989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.094015    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:59210->10.134.0.40:15021: read: connection reset by peer" start-of-body=
Jul 10 02:42:37.094989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.094094    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:59210->10.134.0.40:15021: read: connection reset by peer"
Jul 10 02:42:37.838591 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.838556    2579 generic.go:356] "Generic (PLEG): container finished" podID="93504187-18ea-4991-b85b-e1c6b59c0398" containerID="e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14" exitCode=0
Jul 10 02:42:37.838792 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.838611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerDied","Data":"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"}
Jul 10 02:42:37.839070 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.839056    2579 scope.go:117] "RemoveContainer" containerID="e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"
Jul 10 02:42:37.840098 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.840070    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:37.840154 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.840138    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:37.840189 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:37.840174    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:42:38.845341 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:38.845308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerStarted","Data":"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"}
Jul 10 02:42:39.028572 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:39.028531    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:39.028831 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:39.028807    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 02:42:39.028897 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:39.028869    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 02:42:40.028999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:40.028960    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 02:42:40.029391 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:40.029024    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 02:42:41.029184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:41.029144    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 02:42:41.029592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:41.029216    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 02:42:42.029382 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:42.029344    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Jul 10 02:42:42.029813 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:42.029414    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Jul 10 02:42:43.033111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:43.033081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:43.033541 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:43.033451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:42:43.034225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:42:43.034206    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:44:08.239972 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:08.239939    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:44:08.240577 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:08.240150    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" containerID="cri-o://28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b" gracePeriod=30
Jul 10 02:44:13.036425 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.036383    2579 patch_prober.go:28] interesting pod/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:44:13.036829 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.036440    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:44:13.490801 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.490771    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:44:13.623525 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623421    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623525 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623482    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623525 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623525    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn8qt\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623892 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623551    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623892 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623572    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623892 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623603    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.623892 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623652    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.624123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623957    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:13.624123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.623970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:13.624123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624058    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.624123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token\") pod \"93504187-18ea-4991-b85b-e1c6b59c0398\" (UID: \"93504187-18ea-4991-b85b-e1c6b59c0398\") "
Jul 10 02:44:13.624354 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624130    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:13.624553 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624480    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.624688 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624627    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.624781 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624691    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.624906 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624878    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:44:13.625082 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.624945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data" (OuterVolumeSpecName: "istio-data") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:13.626195 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.626167    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:13.626329 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.626305    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:44:13.626622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.626604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt" (OuterVolumeSpecName: "kube-api-access-kn8qt") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "kube-api-access-kn8qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:44:13.626710 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.626687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token" (OuterVolumeSpecName: "istio-token") pod "93504187-18ea-4991-b85b-e1c6b59c0398" (UID: "93504187-18ea-4991-b85b-e1c6b59c0398"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:44:13.725538 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725505    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.725538 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725533    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn8qt\" (UniqueName: \"kubernetes.io/projected/93504187-18ea-4991-b85b-e1c6b59c0398-kube-api-access-kn8qt\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.725538 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725544    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.725818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725553    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93504187-18ea-4991-b85b-e1c6b59c0398-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.725818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725561    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93504187-18ea-4991-b85b-e1c6b59c0398-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:13.725818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:13.725568    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93504187-18ea-4991-b85b-e1c6b59c0398-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:44:14.321500 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.321469    2579 generic.go:356] "Generic (PLEG): container finished" podID="93504187-18ea-4991-b85b-e1c6b59c0398" containerID="28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b" exitCode=0
Jul 10 02:44:14.321921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.321515    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"
Jul 10 02:44:14.321921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.321549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerDied","Data":"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"}
Jul 10 02:44:14.321921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.321603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv" event={"ID":"93504187-18ea-4991-b85b-e1c6b59c0398","Type":"ContainerDied","Data":"d257784421bb91ae2d3431832841fb5df3ba3d8f66988ed5c58d3b1eef66912d"}
Jul 10 02:44:14.321921 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.321625    2579 scope.go:117] "RemoveContainer" containerID="28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"
Jul 10 02:44:14.330874 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.330854    2579 scope.go:117] "RemoveContainer" containerID="e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"
Jul 10 02:44:14.338972 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.338951    2579 scope.go:117] "RemoveContainer" containerID="28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"
Jul 10 02:44:14.339230 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:44:14.339210    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b\": container with ID starting with 28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b not found: ID does not exist" containerID="28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"
Jul 10 02:44:14.339279 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.339240    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b"} err="failed to get container status \"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b\": rpc error: code = NotFound desc = could not find container \"28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b\": container with ID starting with 28ba02d3946eea870442ca550b775a679d1c04841b623a4dde4b025df21bb36b not found: ID does not exist"
Jul 10 02:44:14.339279 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.339258    2579 scope.go:117] "RemoveContainer" containerID="e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"
Jul 10 02:44:14.339472 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:44:14.339451    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14\": container with ID starting with e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14 not found: ID does not exist" containerID="e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"
Jul 10 02:44:14.339512 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.339479    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14"} err="failed to get container status \"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14\": rpc error: code = NotFound desc = could not find container \"e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14\": container with ID starting with e7e24e55d5d3496f31b7d01c6e569c91716ae864323e47fc2ed43441065d9a14 not found: ID does not exist"
Jul 10 02:44:14.345427 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.345402    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:44:14.351606 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:14.351587    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ad6c59d4-openshift-default-7b896b7d59-9x2tv"]
Jul 10 02:44:15.246050 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:15.246018    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" path="/var/lib/kubelet/pods/93504187-18ea-4991-b85b-e1c6b59c0398/volumes"
Jul 10 02:44:35.267630 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:35.267600    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:44:35.270567 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:44:35.270548    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:45:00.137993 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.137958    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:45:00.138571 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138550    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:00.138676 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138579    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:00.138676 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138615    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:45:00.138676 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138626    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aca65e2-83d4-4806-987e-25c3c4234161" containerName="istio-proxy"
Jul 10 02:45:00.138879 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138760    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:00.138879 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.138779    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:00.142077 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.142057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"
Jul 10 02:45:00.144894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.144872    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2pvrv\""
Jul 10 02:45:00.148716 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.148690    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:45:00.153239 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.153223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"
Jul 10 02:45:00.283536 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.283510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:45:00.285677 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:45:00.285647    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0929cc8_f6bf_4f2d_94d7_20ffeb58e4d6.slice/crio-e43b34238f4098e937d82c4fb0ffcfae83cf708b54572466204613698626775d WatchSource:0}: Error finding container e43b34238f4098e937d82c4fb0ffcfae83cf708b54572466204613698626775d: Status 404 returned error can't find the container with id e43b34238f4098e937d82c4fb0ffcfae83cf708b54572466204613698626775d
Jul 10 02:45:00.558787 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.558751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" event={"ID":"e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6","Type":"ContainerStarted","Data":"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"}
Jul 10 02:45:00.558952 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.558816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" event={"ID":"e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6","Type":"ContainerStarted","Data":"e43b34238f4098e937d82c4fb0ffcfae83cf708b54572466204613698626775d"}
Jul 10 02:45:00.577160 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:00.577116    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" podStartSLOduration=0.577100792 podStartE2EDuration="577.100792ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:00.574856364 +0000 UTC m=+1825.871438143" watchObservedRunningTime="2026-07-10 02:45:00.577100792 +0000 UTC m=+1825.873682566"
Jul 10 02:45:03.784308 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.784270    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:45:03.784927 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.784902    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:03.784927 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.784925    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93504187-18ea-4991-b85b-e1c6b59c0398" containerName="istio-proxy"
Jul 10 02:45:03.788303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.788286    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.790770 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.790724    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2f242d-openshift-default-dockercfg-h6kss\""
Jul 10 02:45:03.799065 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.799043    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:45:03.896904 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.896870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897326 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897402 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897420    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897595 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897595 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.897698 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.897600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw26d\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998816 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998816 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw26d\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.998816 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.998587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.999104 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.999078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.999198 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.999117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.999198 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.999170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.999198 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.999189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:03.999389 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:03.999360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.001106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.001087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.001189 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.001134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.006909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.006890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw26d\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.007000 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.006953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token\") pod \"e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") " pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.099241 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.099142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:04.247011 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.246983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:45:04.248540 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:45:04.248509    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74262c3d_f049_4c1a_afc7_1b334052578b.slice/crio-176ce5366cdce2b8a994621cfaf7882e0450712a6f01a1d890d3d29323f384b0 WatchSource:0}: Error finding container 176ce5366cdce2b8a994621cfaf7882e0450712a6f01a1d890d3d29323f384b0: Status 404 returned error can't find the container with id 176ce5366cdce2b8a994621cfaf7882e0450712a6f01a1d890d3d29323f384b0
Jul 10 02:45:04.251007 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.250970    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:04.251078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.251039    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:04.251130 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.251090    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:04.579688 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.579649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerStarted","Data":"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"}
Jul 10 02:45:04.579908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.579708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerStarted","Data":"176ce5366cdce2b8a994621cfaf7882e0450712a6f01a1d890d3d29323f384b0"}
Jul 10 02:45:04.600658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:04.600609    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podStartSLOduration=1.600590103 podStartE2EDuration="1.600590103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:04.599296445 +0000 UTC m=+1829.895878222" watchObservedRunningTime="2026-07-10 02:45:04.600590103 +0000 UTC m=+1829.897171965"
Jul 10 02:45:05.100053 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:05.100012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:05.104785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:05.104761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:05.584163 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:05.584127    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:05.585031 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:05.585012    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:11.799429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.799395    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:45:11.802927 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.802895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.805711 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.805690    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1238eb-openshift-default-dockercfg-7m7tl\""
Jul 10 02:45:11.814052 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.814024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:45:11.979184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979184 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcmmq\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979288    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979421 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979381    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979588 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:11.979588 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:11.979504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081062 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.080959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081062 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081605 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcmmq\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081605 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081605 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081827 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.081960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.081898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.082435 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.082410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.084549 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.084505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.084877 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.084847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.090127 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.090097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcmmq\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.090127 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.090115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token\") pod \"e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") " pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.115579 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.115432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:12.280250 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.280213    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:45:12.284271 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:45:12.283605    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cf24b9_7a6e_440c_8c66_db03e09fb612.slice/crio-8f30de3f612dcafd6c253a645a1828a7bb054e150366e61449131dcfbd88cfa6 WatchSource:0}: Error finding container 8f30de3f612dcafd6c253a645a1828a7bb054e150366e61449131dcfbd88cfa6: Status 404 returned error can't find the container with id 8f30de3f612dcafd6c253a645a1828a7bb054e150366e61449131dcfbd88cfa6
Jul 10 02:45:12.286432 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.286383    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:12.286521 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.286487    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:12.286562 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.286531    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:12.619081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.618984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerStarted","Data":"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"}
Jul 10 02:45:12.619081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.619054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerStarted","Data":"8f30de3f612dcafd6c253a645a1828a7bb054e150366e61449131dcfbd88cfa6"}
Jul 10 02:45:12.639594 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:12.639537    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podStartSLOduration=1.63952146 podStartE2EDuration="1.63952146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:12.637815918 +0000 UTC m=+1837.934397693" watchObservedRunningTime="2026-07-10 02:45:12.63952146 +0000 UTC m=+1837.936103234"
Jul 10 02:45:13.116091 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:13.116060    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:13.120937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:13.120914    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:13.624070 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:13.624042    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:13.624909 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:13.624891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:21.625903 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:21.625864    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:45:21.626282 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:21.625934    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:45:23.684474 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.684441    2579 generic.go:356] "Generic (PLEG): container finished" podID="74262c3d-f049-4c1a-afc7-1b334052578b" containerID="bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599" exitCode=0
Jul 10 02:45:23.684987 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.684503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerDied","Data":"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"}
Jul 10 02:45:23.685062 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.685045    2579 scope.go:117] "RemoveContainer" containerID="bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"
Jul 10 02:45:23.686448 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.686405    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:23.686787 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.686761    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:23.686962 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:23.686942    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:24.695090 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:24.695046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerStarted","Data":"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"}
Jul 10 02:45:25.100429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:25.100391    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:25.100870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:25.100766    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 10 02:45:25.101062 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:25.101037    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 10 02:45:26.100004 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:26.099962    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 10 02:45:26.100652 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:26.100618    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 10 02:45:27.100141 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:27.100102    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 10 02:45:27.100633 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:27.100173    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 10 02:45:28.100542 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:28.100500    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 10 02:45:28.101069 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:28.100572    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 10 02:45:29.103034 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:29.103001    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:29.103464 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:29.103263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:29.103967 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:29.103951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:45:29.665895 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:29.665856    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:45:29.666072 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:29.665913    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:45:30.737791 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.737757    2579 generic.go:356] "Generic (PLEG): container finished" podID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerID="6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763" exitCode=0
Jul 10 02:45:30.738250 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.737827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerDied","Data":"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"}
Jul 10 02:45:30.738319 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.738305    2579 scope.go:117] "RemoveContainer" containerID="6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"
Jul 10 02:45:30.739300 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.739271    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:30.739393 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.739350    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:30.739457 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:30.739400    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:45:31.745178 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:31.745143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerStarted","Data":"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"}
Jul 10 02:45:32.115712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:32.115668    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:32.115929 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:32.115902    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 10 02:45:32.115979 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:32.115948    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 10 02:45:33.116170 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:33.116125    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 10 02:45:33.116673 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:33.116195    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 10 02:45:34.116539 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:34.116501    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 10 02:45:34.116984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:34.116555    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 10 02:45:35.116323 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:35.116273    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 10 02:45:35.116585 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:35.116341    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 10 02:45:36.119344 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:36.119311    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:36.119793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:36.119754    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:45:36.120429 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:45:36.120411    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:46:00.010791 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:00.010684    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:46:00.011346 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:00.010965    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" podUID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" containerName="cleanup" containerID="cri-o://70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68" gracePeriod=30
Jul 10 02:46:30.226694 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:30.226667    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"
Jul 10 02:46:31.047292 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.047263    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" containerID="70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68" exitCode=137
Jul 10 02:46:31.047544 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.047306    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"
Jul 10 02:46:31.047544 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.047344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" event={"ID":"e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6","Type":"ContainerDied","Data":"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"}
Jul 10 02:46:31.047544 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.047402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7" event={"ID":"e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6","Type":"ContainerDied","Data":"e43b34238f4098e937d82c4fb0ffcfae83cf708b54572466204613698626775d"}
Jul 10 02:46:31.047544 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.047426    2579 scope.go:117] "RemoveContainer" containerID="70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"
Jul 10 02:46:31.059439 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.059404    2579 scope.go:117] "RemoveContainer" containerID="70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"
Jul 10 02:46:31.059926 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:46:31.059894    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68\": container with ID starting with 70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68 not found: ID does not exist" containerID="70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"
Jul 10 02:46:31.060040 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.059931    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68"} err="failed to get container status \"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68\": rpc error: code = NotFound desc = could not find container \"70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68\": container with ID starting with 70dc1cdd91daacc5d91cd11dfb3e6f5224e8f9cb6a7e17be739beb505dd6de68 not found: ID does not exist"
Jul 10 02:46:31.081925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.081891    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:46:31.083259 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.083237    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727525-r8zv7"]
Jul 10 02:46:31.245341 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:46:31.245303    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" path="/var/lib/kubelet/pods/e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6/volumes"
Jul 10 02:47:12.385712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:12.385650    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:47:12.386308 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:12.385947    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" containerID="cri-o://9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9" gracePeriod=30
Jul 10 02:47:14.105329 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:14.105294    2579 patch_prober.go:28] interesting pod/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:47:14.105687 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:14.105353    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:47:17.646202 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.646145    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:47:17.765563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765518    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.765793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.765793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765617    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.765793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765643    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.765793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765694    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw26d\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.765793 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.766085 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765915    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.766085 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.766085 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.765988    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket\") pod \"74262c3d-f049-4c1a-afc7-1b334052578b\" (UID: \"74262c3d-f049-4c1a-afc7-1b334052578b\") "
Jul 10 02:47:17.766244 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766177    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:17.766497 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:17.766497 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766186    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:17.766694 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766473    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:17.766773 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766711    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74262c3d-f049-4c1a-afc7-1b334052578b-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.766773 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766759    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.766875 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766774    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.766875 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.766789    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.767138 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.767117    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data" (OuterVolumeSpecName: "istio-data") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:17.768020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.767998    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:17.768126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.768075    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:47:17.768516 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.768493    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d" (OuterVolumeSpecName: "kube-api-access-fw26d") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "kube-api-access-fw26d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:17.768582 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.768498    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token" (OuterVolumeSpecName: "istio-token") pod "74262c3d-f049-4c1a-afc7-1b334052578b" (UID: "74262c3d-f049-4c1a-afc7-1b334052578b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:17.867631 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.867602    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.867631 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.867628    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw26d\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-kube-api-access-fw26d\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.867631 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.867639    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74262c3d-f049-4c1a-afc7-1b334052578b-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.867882 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.867649    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74262c3d-f049-4c1a-afc7-1b334052578b-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:17.867882 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:17.867656    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74262c3d-f049-4c1a-afc7-1b334052578b-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:18.291015 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.290979    2579 generic.go:356] "Generic (PLEG): container finished" podID="74262c3d-f049-4c1a-afc7-1b334052578b" containerID="9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9" exitCode=0
Jul 10 02:47:18.291187 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.291030    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"
Jul 10 02:47:18.291187 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.291062    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerDied","Data":"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"}
Jul 10 02:47:18.291187 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.291116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8" event={"ID":"74262c3d-f049-4c1a-afc7-1b334052578b","Type":"ContainerDied","Data":"176ce5366cdce2b8a994621cfaf7882e0450712a6f01a1d890d3d29323f384b0"}
Jul 10 02:47:18.291187 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.291138    2579 scope.go:117] "RemoveContainer" containerID="9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"
Jul 10 02:47:18.300746 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.300709    2579 scope.go:117] "RemoveContainer" containerID="bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"
Jul 10 02:47:18.308649 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.308631    2579 scope.go:117] "RemoveContainer" containerID="9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"
Jul 10 02:47:18.308911 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:18.308891    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9\": container with ID starting with 9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9 not found: ID does not exist" containerID="9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"
Jul 10 02:47:18.308986 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.308918    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9"} err="failed to get container status \"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9\": rpc error: code = NotFound desc = could not find container \"9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9\": container with ID starting with 9d4f36b59b22c183767abf2408490c777f6c289f6596db12c82b5de372260fd9 not found: ID does not exist"
Jul 10 02:47:18.308986 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.308933    2579 scope.go:117] "RemoveContainer" containerID="bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"
Jul 10 02:47:18.309157 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:18.309139    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599\": container with ID starting with bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599 not found: ID does not exist" containerID="bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"
Jul 10 02:47:18.309199 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.309162    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599"} err="failed to get container status \"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599\": rpc error: code = NotFound desc = could not find container \"bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599\": container with ID starting with bb50303c5c06edc5083319d0f8292f329e2505bdaaebf24b9ae73de2154f3599 not found: ID does not exist"
Jul 10 02:47:18.313414 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.313392    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:47:18.317848 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:18.317825    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2f242d-openshift-default-6df5d78f45-mlkx8"]
Jul 10 02:47:19.244966 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:19.244936    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" path="/var/lib/kubelet/pods/74262c3d-f049-4c1a-afc7-1b334052578b/volumes"
Jul 10 02:47:19.578270 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:19.578234    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:47:19.578553 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:19.578525    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" containerID="cri-o://665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac" gracePeriod=30
Jul 10 02:47:20.468683 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.468647    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469253    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469277    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469321    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" containerName="cleanup"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469329    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" containerName="cleanup"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469419    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0929cc8-f6bf-4f2d-94d7-20ffeb58e4d6" containerName="cleanup"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469435    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:20.469657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.469447    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:20.472960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.472935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.475478 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.475456    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c90820-openshift-default-dockercfg-j2wzv\""
Jul 10 02:47:20.483957 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.483933    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:20.595374 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.595777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.595777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bw87\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.595777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.595777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.595777 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.596046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.596046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.596046 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.595885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696377 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696563 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bw87\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.696838 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.697053 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.696946    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.697053 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.697027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.697196 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.697172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.697331 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.697294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.697438 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.697418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.698956 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.698939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.699059 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.699038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.704494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.704468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bw87\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.704494 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.704479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token\") pod \"e2e-api-a-c90820-openshift-default-5b66d68887-pf48s\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") " pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.783532 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.783509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:20.927481 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.927451    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:20.928854 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:47:20.928825    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc348dd81_a2e6_4d67_ad41_29379cc5993a.slice/crio-2186e1c2d162ae4379c6bb932b91656f3a12f6147b2aa91144e1ad9d533dbceb WatchSource:0}: Error finding container 2186e1c2d162ae4379c6bb932b91656f3a12f6147b2aa91144e1ad9d533dbceb: Status 404 returned error can't find the container with id 2186e1c2d162ae4379c6bb932b91656f3a12f6147b2aa91144e1ad9d533dbceb
Jul 10 02:47:20.931010 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.930975    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:20.931093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.931048    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:20.931093 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:20.931074    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:21.121944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.121859    2579 patch_prober.go:28] interesting pod/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:47:21.121944 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.121922    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:47:21.319217 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.319174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerStarted","Data":"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"}
Jul 10 02:47:21.319419 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.319269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerStarted","Data":"2186e1c2d162ae4379c6bb932b91656f3a12f6147b2aa91144e1ad9d533dbceb"}
Jul 10 02:47:21.341660 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.341604    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podStartSLOduration=1.341584221 podStartE2EDuration="1.341584221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:21.338445884 +0000 UTC m=+1966.635027658" watchObservedRunningTime="2026-07-10 02:47:21.341584221 +0000 UTC m=+1966.638165998"
Jul 10 02:47:21.783796 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.783760    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:21.788560 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:21.788537    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:22.324796 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:22.324773    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:22.325750 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:22.325711    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:24.821622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.821591    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:47:24.941445 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941367    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941445 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941415    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941620 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941452    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941620 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941558    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941620 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941614    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941784 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941756    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:24.941784 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941772    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:24.941898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941783    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcmmq\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.941898 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941841    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.942006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941899    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.942006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data\") pod \"31cf24b9-7a6e-440c-8c66-db03e09fb612\" (UID: \"31cf24b9-7a6e-440c-8c66-db03e09fb612\") "
Jul 10 02:47:24.942006 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.941975    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:24.942228 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.942201    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:24.942610 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.942588    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:24.942705 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.942616    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:24.942705 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.942632    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31cf24b9-7a6e-440c-8c66-db03e09fb612-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:24.942705 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.942647    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:24.943268 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.943240    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data" (OuterVolumeSpecName: "istio-data") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:24.943908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.943880    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:24.944014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.943953    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:47:24.944014 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.943978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq" (OuterVolumeSpecName: "kube-api-access-jcmmq") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "kube-api-access-jcmmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:24.944097 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:24.944061    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token" (OuterVolumeSpecName: "istio-token") pod "31cf24b9-7a6e-440c-8c66-db03e09fb612" (UID: "31cf24b9-7a6e-440c-8c66-db03e09fb612"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:25.043474 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.043449    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcmmq\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-kube-api-access-jcmmq\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:25.043474 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.043475    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:25.043624 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.043486    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:25.043624 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.043494    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:25.043624 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.043501    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31cf24b9-7a6e-440c-8c66-db03e09fb612-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:25.340939 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.340904    2579 generic.go:356] "Generic (PLEG): container finished" podID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerID="665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac" exitCode=0
Jul 10 02:47:25.341106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.340948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerDied","Data":"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"}
Jul 10 02:47:25.341106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.340971    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"
Jul 10 02:47:25.341106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.341006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22" event={"ID":"31cf24b9-7a6e-440c-8c66-db03e09fb612","Type":"ContainerDied","Data":"8f30de3f612dcafd6c253a645a1828a7bb054e150366e61449131dcfbd88cfa6"}
Jul 10 02:47:25.341106 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.341028    2579 scope.go:117] "RemoveContainer" containerID="665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"
Jul 10 02:47:25.350028 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.350007    2579 scope.go:117] "RemoveContainer" containerID="6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"
Jul 10 02:47:25.358948 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.358928    2579 scope.go:117] "RemoveContainer" containerID="665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"
Jul 10 02:47:25.359280 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:25.359252    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac\": container with ID starting with 665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac not found: ID does not exist" containerID="665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"
Jul 10 02:47:25.359380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.359291    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac"} err="failed to get container status \"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac\": rpc error: code = NotFound desc = could not find container \"665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac\": container with ID starting with 665c1a86430747565fddd4e03532b2b15971bfaed9a5afd9e7544e509aa7b4ac not found: ID does not exist"
Jul 10 02:47:25.359380 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.359317    2579 scope.go:117] "RemoveContainer" containerID="6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"
Jul 10 02:47:25.359832 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:25.359806    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763\": container with ID starting with 6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763 not found: ID does not exist" containerID="6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"
Jul 10 02:47:25.359994 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.359970    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763"} err="failed to get container status \"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763\": rpc error: code = NotFound desc = could not find container \"6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763\": container with ID starting with 6745a82d284d41e0edfa46ebc0d02e94e6c0bca0ee0dba5dabffd6a39e1ae763 not found: ID does not exist"
Jul 10 02:47:25.362565 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.362546    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:47:25.366146 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:25.366124    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1238eb-openshift-default-58c74bfcc-mdq22"]
Jul 10 02:47:27.244606 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:27.244572    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" path="/var/lib/kubelet/pods/31cf24b9-7a6e-440c-8c66-db03e09fb612/volumes"
Jul 10 02:47:28.553084 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553053    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:47:28.553684 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553667    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:28.553752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553689    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74262c3d-f049-4c1a-afc7-1b334052578b" containerName="istio-proxy"
Jul 10 02:47:28.553752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553700    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:28.553752 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553708    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:28.553914 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553900    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:28.553950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.553918    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:28.557510 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.557491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.566081 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.566046    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f312f7-openshift-default-dockercfg-kdccb\""
Jul 10 02:47:28.569218 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.569188    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:47:28.681119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681221    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9kzx\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681333 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681498 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.681498 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.681371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.782845 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.782809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783018 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.782870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783018 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.782919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783018 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.782955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783249 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9kzx\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783249 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783249 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783072    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783249 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783249 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783501 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783249    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783501 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783501 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783696 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.783840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.783820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.785425 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.785404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.785677 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.785659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.791170 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.791146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.791775 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.791753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9kzx\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx\") pod \"e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") " pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:28.878577 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:28.878483    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:29.023619 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.023588    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:47:29.025589 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:47:29.025562    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec17a90_6ebc_40f4_824d_6e8b57223f13.slice/crio-1835fe9f79be28058f7db95db1d2314139f830d3ea1ecf5e5caae792fc0b495a WatchSource:0}: Error finding container 1835fe9f79be28058f7db95db1d2314139f830d3ea1ecf5e5caae792fc0b495a: Status 404 returned error can't find the container with id 1835fe9f79be28058f7db95db1d2314139f830d3ea1ecf5e5caae792fc0b495a
Jul 10 02:47:29.027941 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.027902    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:29.028016 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.027969    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:29.028016 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.027996    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:29.365224 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.365186    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerStarted","Data":"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"}
Jul 10 02:47:29.365372 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.365244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerStarted","Data":"1835fe9f79be28058f7db95db1d2314139f830d3ea1ecf5e5caae792fc0b495a"}
Jul 10 02:47:29.386751 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.386683    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podStartSLOduration=1.386667737 podStartE2EDuration="1.386667737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:29.385873753 +0000 UTC m=+1974.682455561" watchObservedRunningTime="2026-07-10 02:47:29.386667737 +0000 UTC m=+1974.683249512"
Jul 10 02:47:29.878915 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:29.878885    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:30.884129 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:30.884098    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:31.375818 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:31.375786    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:31.376930 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:31.376913    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:38.365904 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:38.365861    2579 patch_prober.go:28] interesting pod/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:47:38.366294 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:38.365972    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:47:40.434412 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.434376    2579 generic.go:356] "Generic (PLEG): container finished" podID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerID="36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0" exitCode=0
Jul 10 02:47:40.434871 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.434451    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerDied","Data":"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"}
Jul 10 02:47:40.434871 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.434863    2579 scope.go:117] "RemoveContainer" containerID="36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"
Jul 10 02:47:40.435891 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.435859    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:40.436001 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.435951    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:40.436065 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:40.436000    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:41.441984 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:41.441950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerStarted","Data":"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"}
Jul 10 02:47:41.783937 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:41.783910    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:41.784178 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:41.784158    2579 patch_prober.go:28] interesting pod/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 02:47:41.784225 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:41.784212    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 02:47:42.784213 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:42.784173    2579 patch_prober.go:28] interesting pod/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 02:47:42.784656 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:42.784226    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 02:47:43.784965 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:43.784928    2579 patch_prober.go:28] interesting pod/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 02:47:43.785655 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:43.785624    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 02:47:44.784598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:44.784563    2579 patch_prober.go:28] interesting pod/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 10 02:47:44.784882 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:44.784638    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 10 02:47:45.788166 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:45.788137    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:45.788640 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:45.788413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:45.789136 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:45.789117    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:47.190227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.189152    2579 patch_prober.go:28] interesting pod/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:57692->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Jul 10 02:47:47.190227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.189213    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:57692->10.134.0.45:15021: read: connection reset by peer"
Jul 10 02:47:47.474471 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.474357    2579 generic.go:356] "Generic (PLEG): container finished" podID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerID="1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234" exitCode=0
Jul 10 02:47:47.474471 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.474434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerDied","Data":"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"}
Jul 10 02:47:47.474993 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.474974    2579 scope.go:117] "RemoveContainer" containerID="1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"
Jul 10 02:47:47.476017 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.475990    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:47.476112 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.476057    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:47.476112 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:47.476103    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:48.481490 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:48.481457    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerStarted","Data":"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"}
Jul 10 02:47:48.879074 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:48.879043    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:48.879248 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:48.879218    2579 patch_prober.go:28] interesting pod/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 02:47:48.879292 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:48.879262    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 02:47:49.878968 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:49.878927    2579 patch_prober.go:28] interesting pod/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 02:47:49.879364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:49.878994    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 02:47:50.879869 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:50.879828    2579 patch_prober.go:28] interesting pod/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 02:47:50.880252 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:50.879904    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 02:47:50.974642 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:50.974602    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:50.974917 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:50.974870    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy" containerID="cri-o://ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2" gracePeriod=30
Jul 10 02:47:51.879585 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:51.879542    2579 patch_prober.go:28] interesting pod/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 10 02:47:51.879823 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:51.879630    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 10 02:47:52.882870 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:52.882835    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:52.883338 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:52.883225    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:52.883830 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:52.883809    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:47:56.233232 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.233204    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:56.383785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.383989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383799    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.383989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383861    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.383989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.383989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383908    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.383989 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bw87\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.384260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.383998    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.384260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384086    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.384260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo\") pod \"c348dd81-a2e6-4d67-ad41-29379cc5993a\" (UID: \"c348dd81-a2e6-4d67-ad41-29379cc5993a\") "
Jul 10 02:47:56.384260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384125    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:56.384260 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:56.384523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384365    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:56.384523 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384405    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:56.384625 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384535    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.384625 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384555    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.384625 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384569    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.384625 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384583    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c348dd81-a2e6-4d67-ad41-29379cc5993a-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.384998 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.384977    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:56.386240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.386214    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:56.386472 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.386448    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:47:56.386768 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.386747    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:56.386768 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.386746    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87" (OuterVolumeSpecName: "kube-api-access-4bw87") pod "c348dd81-a2e6-4d67-ad41-29379cc5993a" (UID: "c348dd81-a2e6-4d67-ad41-29379cc5993a"). InnerVolumeSpecName "kube-api-access-4bw87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:56.485950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.485920    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.485950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.485949    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.485950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.485957    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.486180 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.485966    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c348dd81-a2e6-4d67-ad41-29379cc5993a-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.486180 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.485974    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bw87\" (UniqueName: \"kubernetes.io/projected/c348dd81-a2e6-4d67-ad41-29379cc5993a-kube-api-access-4bw87\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:47:56.530932 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.530901    2579 generic.go:356] "Generic (PLEG): container finished" podID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerID="ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2" exitCode=0
Jul 10 02:47:56.531080 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.530952    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"
Jul 10 02:47:56.531080 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.530969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerDied","Data":"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"}
Jul 10 02:47:56.531080 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.531013    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s" event={"ID":"c348dd81-a2e6-4d67-ad41-29379cc5993a","Type":"ContainerDied","Data":"2186e1c2d162ae4379c6bb932b91656f3a12f6147b2aa91144e1ad9d533dbceb"}
Jul 10 02:47:56.531080 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.531034    2579 scope.go:117] "RemoveContainer" containerID="ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"
Jul 10 02:47:56.540605 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.540589    2579 scope.go:117] "RemoveContainer" containerID="36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"
Jul 10 02:47:56.548310 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.548293    2579 scope.go:117] "RemoveContainer" containerID="ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"
Jul 10 02:47:56.548579 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:56.548561    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2\": container with ID starting with ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2 not found: ID does not exist" containerID="ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"
Jul 10 02:47:56.548626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.548586    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2"} err="failed to get container status \"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2\": rpc error: code = NotFound desc = could not find container \"ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2\": container with ID starting with ef115247939bd0cdeed41dbbca6ea8789832a88aa1946859f5f3a0400121f3c2 not found: ID does not exist"
Jul 10 02:47:56.548626 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.548601    2579 scope.go:117] "RemoveContainer" containerID="36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"
Jul 10 02:47:56.548822 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:47:56.548805    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0\": container with ID starting with 36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0 not found: ID does not exist" containerID="36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"
Jul 10 02:47:56.548861 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.548827    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0"} err="failed to get container status \"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0\": rpc error: code = NotFound desc = could not find container \"36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0\": container with ID starting with 36bc84df793f57ad9a31bc400ab8c206ece8fb175efffa77197149c8a4ffc0f0 not found: ID does not exist"
Jul 10 02:47:56.553670 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.553645    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:56.555531 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:56.555512    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c90820-openshift-default-5b66d68887-pf48s"]
Jul 10 02:47:57.245068 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:57.245033    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" path="/var/lib/kubelet/pods/c348dd81-a2e6-4d67-ad41-29379cc5993a/volumes"
Jul 10 02:47:58.155266 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:58.155235    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:47:58.155502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:58.155423    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy" containerID="cri-o://ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc" gracePeriod=30
Jul 10 02:47:59.048608 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.048571    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:47:59.049173 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049153    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:47:59.049258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049177    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:47:59.049258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049218    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:59.049258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049227    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31cf24b9-7a6e-440c-8c66-db03e09fb612" containerName="istio-proxy"
Jul 10 02:47:59.049412 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049333    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:47:59.049412 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.049350    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:47:59.052956 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.052933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.055640 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.055618    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e239ac-openshift-default-dockercfg-9dbx2\""
Jul 10 02:47:59.068137 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.067761    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:47:59.210388 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210585 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210585 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210551    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210695 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2s4k\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210695 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210695 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210724    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.210840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.210766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.311810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.311810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.311810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.311983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312111 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2s4k\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312339 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.312679 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.312599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.314982 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.314958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.315193 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.315175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.319938 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.319915    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.320221 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.320203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2s4k\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k\") pod \"e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") " pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.369192 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.369163    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:47:59.509386 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.509359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:47:59.511395 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:47:59.511370    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7351c9c5_c0f2_4fee_9c7f_47424d98fc50.slice/crio-642eeb16465b529c556549199fbe215fea25e63d07368521d2c1d5380348397f WatchSource:0}: Error finding container 642eeb16465b529c556549199fbe215fea25e63d07368521d2c1d5380348397f: Status 404 returned error can't find the container with id 642eeb16465b529c556549199fbe215fea25e63d07368521d2c1d5380348397f
Jul 10 02:47:59.513557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.513524    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:59.513646 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.513591    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:59.513646 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.513620    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:47:59.551743 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:47:59.551685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerStarted","Data":"642eeb16465b529c556549199fbe215fea25e63d07368521d2c1d5380348397f"}
Jul 10 02:48:00.558197 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:00.558155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerStarted","Data":"5438530ca99ed40ae41fb9255336b88ee755fe029f8d48f7bd941e928ae6f813"}
Jul 10 02:48:00.579932 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:00.579888    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podStartSLOduration=1.579874614 podStartE2EDuration="1.579874614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:00.577554848 +0000 UTC m=+2005.874136661" watchObservedRunningTime="2026-07-10 02:48:00.579874614 +0000 UTC m=+2005.876456389"
Jul 10 02:48:01.369312 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:01.369276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:01.374551 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:01.374525    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:01.563322 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:01.563285    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:01.564284 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:01.564266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:03.419123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.419063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:48:03.553110 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553079    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553110 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553117    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553257    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553379 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553370    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9kzx\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553376    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553458    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553503    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.553557 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data\") pod \"bec17a90-6ebc-40f4-824d-6e8b57223f13\" (UID: \"bec17a90-6ebc-40f4-824d-6e8b57223f13\") "
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553610    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553840    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.553985    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.554003    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.554017    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.554096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.554034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:48:03.554573 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.554553    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data" (OuterVolumeSpecName: "istio-data") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:03.555629 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.555608    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:48:03.555761 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.555724    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:03.556066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.556034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx" (OuterVolumeSpecName: "kube-api-access-n9kzx") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "kube-api-access-n9kzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:03.556066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.556047    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token" (OuterVolumeSpecName: "istio-token") pod "bec17a90-6ebc-40f4-824d-6e8b57223f13" (UID: "bec17a90-6ebc-40f4-824d-6e8b57223f13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:03.576561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.576533    2579 generic.go:356] "Generic (PLEG): container finished" podID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerID="ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc" exitCode=0
Jul 10 02:48:03.576691 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.576584    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"
Jul 10 02:48:03.576691 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.576613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerDied","Data":"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"}
Jul 10 02:48:03.576691 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.576667    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5" event={"ID":"bec17a90-6ebc-40f4-824d-6e8b57223f13","Type":"ContainerDied","Data":"1835fe9f79be28058f7db95db1d2314139f830d3ea1ecf5e5caae792fc0b495a"}
Jul 10 02:48:03.576691 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.576691    2579 scope.go:117] "RemoveContainer" containerID="ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"
Jul 10 02:48:03.586708 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.586691    2579 scope.go:117] "RemoveContainer" containerID="1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"
Jul 10 02:48:03.594894 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.594870    2579 scope.go:117] "RemoveContainer" containerID="ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"
Jul 10 02:48:03.595157 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:48:03.595138    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc\": container with ID starting with ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc not found: ID does not exist" containerID="ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"
Jul 10 02:48:03.595213 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.595166    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc"} err="failed to get container status \"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc\": rpc error: code = NotFound desc = could not find container \"ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc\": container with ID starting with ef3622edfbcb2c9ef5c5a10e7aec994432eedb11cc966aa04795a618fafdefdc not found: ID does not exist"
Jul 10 02:48:03.595213 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.595184    2579 scope.go:117] "RemoveContainer" containerID="1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"
Jul 10 02:48:03.595442 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:48:03.595424    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234\": container with ID starting with 1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234 not found: ID does not exist" containerID="1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"
Jul 10 02:48:03.595497 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.595450    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234"} err="failed to get container status \"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234\": rpc error: code = NotFound desc = could not find container \"1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234\": container with ID starting with 1d1e5e7a320e0b079d53427ad11a7a3a26a0460aa4ef0cb9a0e9f022920b7234 not found: ID does not exist"
Jul 10 02:48:03.598436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.598415    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:48:03.604676 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.604647    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f312f7-openshift-default-67f6dc755-t9gw5"]
Jul 10 02:48:03.654593 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654563    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9kzx\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-kube-api-access-n9kzx\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.654593 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654592    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bec17a90-6ebc-40f4-824d-6e8b57223f13-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.654802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654605    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.654802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654613    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.654802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654621    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:03.654802 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:03.654628    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bec17a90-6ebc-40f4-824d-6e8b57223f13-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:48:05.245524 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:05.245491    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" path="/var/lib/kubelet/pods/bec17a90-6ebc-40f4-824d-6e8b57223f13/volumes"
Jul 10 02:48:07.668764 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.668705    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:48:07.669348 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669325    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:48:07.669348 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669348    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c348dd81-a2e6-4d67-ad41-29379cc5993a" containerName="istio-proxy"
Jul 10 02:48:07.669514 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669365    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.669514 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669373    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.669514 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669418    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.669514 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669428    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.669708 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669540    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.669708 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.669558    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec17a90-6ebc-40f4-824d-6e8b57223f13" containerName="istio-proxy"
Jul 10 02:48:07.671861 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.671838    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.674282 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.674262    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e812a4-openshift-default-dockercfg-hlj9t\""
Jul 10 02:48:07.682610 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.682587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:48:07.690660 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690635    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.690797 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.690797 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.690797 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.690797 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690786    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.691037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.691037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.691037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588k4\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.691037 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.690937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588k4\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.791935 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.791966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.792146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792233 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.792176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792394 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.792361    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792394 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.792371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.792893 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.792871    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.794332 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.794309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.794490 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.794470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.800020 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.799993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.800277 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.800257    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-588k4\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4\") pod \"e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") " pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:07.983960 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:07.983880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:08.124159 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.124123    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:48:08.128365 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:48:08.128333    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44883dda_6118_4430_b867_1ffd4e596b5b.slice/crio-d84f2ac46208c95a482735acc93252760ccdaf035f2b46e6884a7e0687e66c7a WatchSource:0}: Error finding container d84f2ac46208c95a482735acc93252760ccdaf035f2b46e6884a7e0687e66c7a: Status 404 returned error can't find the container with id d84f2ac46208c95a482735acc93252760ccdaf035f2b46e6884a7e0687e66c7a
Jul 10 02:48:08.130445 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.130408    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:08.130549 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.130480    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:08.130549 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.130508    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:08.604441 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.604400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerStarted","Data":"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"}
Jul 10 02:48:08.604639 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.604459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerStarted","Data":"d84f2ac46208c95a482735acc93252760ccdaf035f2b46e6884a7e0687e66c7a"}
Jul 10 02:48:08.625336 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.625274    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podStartSLOduration=1.6252534939999999 podStartE2EDuration="1.625253494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:08.624570815 +0000 UTC m=+2013.921152591" watchObservedRunningTime="2026-07-10 02:48:08.625253494 +0000 UTC m=+2013.921835270"
Jul 10 02:48:08.984248 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:08.984153    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:09.996342 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:09.996284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:10.621282 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:10.621201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:10.622451 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:10.622412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:16.783520 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:16.783119    2579 patch_prober.go:28] interesting pod/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:48940->10.134.0.46:15021: read: connection reset by peer" start-of-body=
Jul 10 02:48:16.783520 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:16.783202    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": read tcp 10.134.0.2:48940->10.134.0.46:15021: read: connection reset by peer"
Jul 10 02:48:17.662066 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.662028    2579 generic.go:356] "Generic (PLEG): container finished" podID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerID="5438530ca99ed40ae41fb9255336b88ee755fe029f8d48f7bd941e928ae6f813" exitCode=0
Jul 10 02:48:17.662246 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.662102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerDied","Data":"5438530ca99ed40ae41fb9255336b88ee755fe029f8d48f7bd941e928ae6f813"}
Jul 10 02:48:17.662528 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.662514    2579 scope.go:117] "RemoveContainer" containerID="5438530ca99ed40ae41fb9255336b88ee755fe029f8d48f7bd941e928ae6f813"
Jul 10 02:48:17.663532 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.663504    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:17.663633 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.663571    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:17.663633 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:17.663613    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:18.668506 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:18.668473    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerStarted","Data":"afba8f8d263251bcbde38a3414e55d6a3017a8201c3349560cd63b8a81391aa1"}
Jul 10 02:48:19.370240 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:19.370207    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:19.370444 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:19.370421    2579 patch_prober.go:28] interesting pod/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 10 02:48:19.370493 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:19.370471    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 10 02:48:20.370327 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:20.370284    2579 patch_prober.go:28] interesting pod/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 10 02:48:20.370754 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:20.370368    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 10 02:48:21.370041 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:21.370002    2579 patch_prober.go:28] interesting pod/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 10 02:48:21.370234 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:21.370075    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 10 02:48:22.370607 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:22.370563    2579 patch_prober.go:28] interesting pod/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 10 02:48:22.371038 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:22.370622    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 10 02:48:23.374304 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.374223    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:23.374914 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.374642    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:23.375467 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.375440    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:48:23.700720 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.700627    2579 generic.go:356] "Generic (PLEG): container finished" podID="44883dda-6118-4430-b867-1ffd4e596b5b" containerID="0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1" exitCode=0
Jul 10 02:48:23.700720 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.700698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerDied","Data":"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"}
Jul 10 02:48:23.701282 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.701268    2579 scope.go:117] "RemoveContainer" containerID="0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"
Jul 10 02:48:23.702216 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.702193    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:23.702266 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.702254    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:23.702301 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:23.702286    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Jul 10 02:48:24.712021 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:24.711983    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerStarted","Data":"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"}
Jul 10 02:48:24.984436 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:24.984344    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:24.985769 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:24.985714    2579 patch_prober.go:28] interesting pod/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:48:24.986017 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:24.985990    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 10 02:48:25.622206 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:25.622166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:25.984977 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:25.984883    2579 patch_prober.go:28] interesting pod/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:48:25.985589 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:25.985555    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 10 02:48:26.984296 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:26.984263    2579 patch_prober.go:28] interesting pod/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:48:26.984486 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:26.984318    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 10 02:48:27.984657 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:27.984619    2579 patch_prober.go:28] interesting pod/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:48:27.985160 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:27.984682    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 10 02:48:28.988088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:28.988059    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:48:28.989280 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:48:28.989261    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:49:35.301147 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:49:35.301116    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:49:35.305433 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:49:35.305411    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:54:35.335624 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:54:35.335593    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:54:35.342139 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:54:35.342120    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qhttr_4ed0745b-06d8-4e46-8b07-9dbc3bead91c/ovn-acl-logging/0.log"
Jul 10 02:55:31.736819 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:31.736782    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:55:31.737243 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:31.737100    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy" containerID="cri-o://afba8f8d263251bcbde38a3414e55d6a3017a8201c3349560cd63b8a81391aa1" gracePeriod=30
Jul 10 02:55:36.875258 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:36.875219    2579 generic.go:356] "Generic (PLEG): container finished" podID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerID="afba8f8d263251bcbde38a3414e55d6a3017a8201c3349560cd63b8a81391aa1" exitCode=0
Jul 10 02:55:36.875677 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:36.875289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerDied","Data":"afba8f8d263251bcbde38a3414e55d6a3017a8201c3349560cd63b8a81391aa1"}
Jul 10 02:55:36.875677 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:36.875367    2579 scope.go:117] "RemoveContainer" containerID="5438530ca99ed40ae41fb9255336b88ee755fe029f8d48f7bd941e928ae6f813"
Jul 10 02:55:36.988958 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:36.988933    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:55:37.035568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035539    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035585    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035618    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035712 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035690    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035916 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035752    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035916 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035787    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035916 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035822    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.035916 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035886    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2s4k\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.036126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo\") pod \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\" (UID: \"7351c9c5-c0f2-4fee-9c7f-47424d98fc50\") "
Jul 10 02:55:37.036126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.035881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:37.036126 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.036060    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:37.036364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.036325    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.036364 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.036351    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.036878 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.036845    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data" (OuterVolumeSpecName: "istio-data") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:37.037209 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.037182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:55:37.037209 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.037196    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:37.038226 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.038189    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:37.038353 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.038327    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k" (OuterVolumeSpecName: "kube-api-access-n2s4k") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "kube-api-access-n2s4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:37.038475 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.038451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token" (OuterVolumeSpecName: "istio-token") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:37.038578 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.038559    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7351c9c5-c0f2-4fee-9c7f-47424d98fc50" (UID: "7351c9c5-c0f2-4fee-9c7f-47424d98fc50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:55:37.137458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137418    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137450    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137462    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137697 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137474    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2s4k\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-kube-api-access-n2s4k\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137697 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137487    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137697 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137500    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.137697 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.137511    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7351c9c5-c0f2-4fee-9c7f-47424d98fc50-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:37.882119 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.882081    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"
Jul 10 02:55:37.882587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.882139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd" event={"ID":"7351c9c5-c0f2-4fee-9c7f-47424d98fc50","Type":"ContainerDied","Data":"642eeb16465b529c556549199fbe215fea25e63d07368521d2c1d5380348397f"}
Jul 10 02:55:37.882587 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.882215    2579 scope.go:117] "RemoveContainer" containerID="afba8f8d263251bcbde38a3414e55d6a3017a8201c3349560cd63b8a81391aa1"
Jul 10 02:55:37.903088 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.903062    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:55:37.907244 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:37.907222    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e239ac-openshift-default-8759b98f9-4z5qd"]
Jul 10 02:55:38.928134 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:38.928105    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:55:38.928603 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:38.928342    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" containerID="cri-o://8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883" gracePeriod=30
Jul 10 02:55:39.244923 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:39.244829    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" path="/var/lib/kubelet/pods/7351c9c5-c0f2-4fee-9c7f-47424d98fc50/volumes"
Jul 10 02:55:43.989716 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:43.989685    2579 patch_prober.go:28] interesting pod/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:55:43.990089 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:43.989763    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 10 02:55:44.181763 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.181721    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:55:44.308417 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308388    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.308583 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308444    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-588k4\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.308583 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308475    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.308583 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308506    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.308583 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308530    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308704    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308761    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs\") pod \"44883dda-6118-4430-b867-1ffd4e596b5b\" (UID: \"44883dda-6118-4430-b867-1ffd4e596b5b\") "
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.308974    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309072    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309262    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309540    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44883dda-6118-4430-b867-1ffd4e596b5b-istiod-ca-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309561    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-credential-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309577    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309592    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-workload-socket\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.310086 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.309939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:44.310963 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.310940    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:44.311053 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.311037    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:44.311123 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.311100    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4" (OuterVolumeSpecName: "kube-api-access-588k4") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "kube-api-access-588k4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:44.311212 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.311195    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44883dda-6118-4430-b867-1ffd4e596b5b" (UID: "44883dda-6118-4430-b867-1ffd4e596b5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:55:44.410918 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.410881    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44883dda-6118-4430-b867-1ffd4e596b5b-istio-podinfo\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.410918 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.410910    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-data\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.410918 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.410920    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44883dda-6118-4430-b867-1ffd4e596b5b-istio-envoy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.411133 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.410928    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-588k4\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-kube-api-access-588k4\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.411133 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.410939    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44883dda-6118-4430-b867-1ffd4e596b5b-istio-token\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\""
Jul 10 02:55:44.920991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.920959    2579 generic.go:356] "Generic (PLEG): container finished" podID="44883dda-6118-4430-b867-1ffd4e596b5b" containerID="8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883" exitCode=0
Jul 10 02:55:44.921172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.921010    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"
Jul 10 02:55:44.921172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.921019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerDied","Data":"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"}
Jul 10 02:55:44.921172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.921063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8" event={"ID":"44883dda-6118-4430-b867-1ffd4e596b5b","Type":"ContainerDied","Data":"d84f2ac46208c95a482735acc93252760ccdaf035f2b46e6884a7e0687e66c7a"}
Jul 10 02:55:44.921172 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.921083    2579 scope.go:117] "RemoveContainer" containerID="8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"
Jul 10 02:55:44.930883 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.930867    2579 scope.go:117] "RemoveContainer" containerID="0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"
Jul 10 02:55:44.939064 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.939043    2579 scope.go:117] "RemoveContainer" containerID="8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"
Jul 10 02:55:44.939329 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:55:44.939309    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883\": container with ID starting with 8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883 not found: ID does not exist" containerID="8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"
Jul 10 02:55:44.939397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.939341    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883"} err="failed to get container status \"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883\": rpc error: code = NotFound desc = could not find container \"8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883\": container with ID starting with 8d22b4e37821c141985df76a3f0c11e5ec171d83fc50f9ecff76d6a0e104c883 not found: ID does not exist"
Jul 10 02:55:44.939397 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.939365    2579 scope.go:117] "RemoveContainer" containerID="0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"
Jul 10 02:55:44.939653 ip-10-0-140-223 kubenswrapper[2579]: E0710 02:55:44.939635    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1\": container with ID starting with 0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1 not found: ID does not exist" containerID="0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"
Jul 10 02:55:44.939692 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.939660    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1"} err="failed to get container status \"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1\": rpc error: code = NotFound desc = could not find container \"0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1\": container with ID starting with 0afa26a68dc1e9963061b49782d50193648369ece6d5b24e035690ee7b2145b1 not found: ID does not exist"
Jul 10 02:55:44.944503 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.944481    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:55:44.947976 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:44.947954    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e812a4-openshift-default-7d5994bcd7-bm8m8"]
Jul 10 02:55:45.215924 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:45.215819    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vprs7_624c704a-7936-4aa0-9c76-e5f9080f2209/server/0.log"
Jul 10 02:55:45.245546 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:45.245511    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" path="/var/lib/kubelet/pods/44883dda-6118-4430-b867-1ffd4e596b5b/volumes"
Jul 10 02:55:45.471357 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:45.471285    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-2f4rw_3435c26f-6547-4282-bc9f-f0117d64ee06/copy-manifests/0.log"
Jul 10 02:55:45.484278 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:45.484258    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-2f4rw_3435c26f-6547-4282-bc9f-f0117d64ee06/manager/0.log"
Jul 10 02:55:48.801367 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:48.801318    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjn9c_e3cbfcff-26cb-4ac3-8f89-245707fab97c/istio-proxy/0.log"
Jul 10 02:55:48.929665 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:48.929628    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmf6d_c93ea990-d141-4a68-90cc-f07a1e17228c/discovery/0.log"
Jul 10 02:55:49.291342 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:49.291314    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/1.log"
Jul 10 02:55:50.028851 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:50.028823    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mzgdt_d6042d4a-8779-4b3d-934f-2263a9145ca4/main/0.log"
Jul 10 02:55:50.268554 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:50.268519    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pcqhz_4fbc297e-c1c3-4afe-8b26-3f98a82c41f5/main/0.log"
Jul 10 02:55:57.593513 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:57.593487    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f5w9n_38b8e300-2322-41e9-830d-0f0fe72835b8/global-pull-secret-syncer/0.log"
Jul 10 02:55:57.715239 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:57.715199    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c5g6r_afafc254-8fa2-4876-9789-425e1c2a8523/konnectivity-agent/0.log"
Jul 10 02:55:57.792579 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:55:57.792535    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-223.ec2.internal_16a2c0a00cdbe9d77d0bff3f91f1c3e7/haproxy/0.log"
Jul 10 02:56:04.752227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.752188    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/alertmanager/0.log"
Jul 10 02:56:04.777773 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.777709    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/config-reloader/0.log"
Jul 10 02:56:04.798510 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.798487    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/kube-rbac-proxy-web/0.log"
Jul 10 02:56:04.821608 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.821583    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/kube-rbac-proxy/0.log"
Jul 10 02:56:04.843157 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.843134    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/kube-rbac-proxy-metric/0.log"
Jul 10 02:56:04.867560 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.867490    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/prom-label-proxy/0.log"
Jul 10 02:56:04.892124 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.892106    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_80c524ec-e303-43c4-b224-9a7a8f1ac2c8/init-config-reloader/0.log"
Jul 10 02:56:04.967622 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.967596    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-state-metrics/0.log"
Jul 10 02:56:04.992509 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:04.992491    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-rbac-proxy-main/0.log"
Jul 10 02:56:05.012900 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.012875    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-4kx7x_61d636a1-904c-4677-acc9-4dd9aa55aa40/kube-rbac-proxy-self/0.log"
Jul 10 02:56:05.039458 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.039439    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7dff7d7994-dkkl7_01c18038-ca2e-40c2-85ab-40cc7caa12e8/metrics-server/0.log"
Jul 10 02:56:05.080210 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.080152    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-z8vs7_5e1dfb10-c355-4395-978b-c895bab566ed/monitoring-plugin/0.log"
Jul 10 02:56:05.111035 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.111010    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/node-exporter/0.log"
Jul 10 02:56:05.130475 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.130392    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/kube-rbac-proxy/0.log"
Jul 10 02:56:05.149991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.149971    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8sb8m_d25ba6ad-0cd9-40da-be81-51109673f2bb/init-textfile/0.log"
Jul 10 02:56:05.398925 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.398817    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/kube-rbac-proxy-main/0.log"
Jul 10 02:56:05.424739 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.424711    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/kube-rbac-proxy-self/0.log"
Jul 10 02:56:05.452658 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.452610    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9vmqp_654697c0-2dad-4b00-bb43-7d2a826e76be/openshift-state-metrics/0.log"
Jul 10 02:56:05.506839 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.506793    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/prometheus/0.log"
Jul 10 02:56:05.527713 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.527689    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/config-reloader/0.log"
Jul 10 02:56:05.548950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.548933    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/thanos-sidecar/0.log"
Jul 10 02:56:05.575320 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.575301    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/kube-rbac-proxy-web/0.log"
Jul 10 02:56:05.599908 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.599890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/kube-rbac-proxy/0.log"
Jul 10 02:56:05.623445 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.623426    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/kube-rbac-proxy-thanos/0.log"
Jul 10 02:56:05.645475 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.645453    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6a51ba4c-143a-427f-825c-060de64f6c11/init-config-reloader/0.log"
Jul 10 02:56:05.685082 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.684996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rrpkh_c1f85e19-00b6-4576-bd54-afca38e59c15/prometheus-operator/0.log"
Jul 10 02:56:05.703976 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.703952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rrpkh_c1f85e19-00b6-4576-bd54-afca38e59c15/kube-rbac-proxy/0.log"
Jul 10 02:56:05.728191 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.728165    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-pvphl_2f3ff963-ff77-447a-8642-42edd4a71ef3/prometheus-operator-admission-webhook/0.log"
Jul 10 02:56:05.762167 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.762142    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/telemeter-client/0.log"
Jul 10 02:56:05.783134 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.783110    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/reload/0.log"
Jul 10 02:56:05.803328 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.803312    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f67b6fd68-ll6wd_a5eee088-9a6b-4c0f-8edb-dfe5f11aaeef/kube-rbac-proxy/0.log"
Jul 10 02:56:05.842016 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.841993    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/thanos-query/0.log"
Jul 10 02:56:05.864096 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.864077    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-web/0.log"
Jul 10 02:56:05.885487 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.885465    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy/0.log"
Jul 10 02:56:05.906356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.906329    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/prom-label-proxy/0.log"
Jul 10 02:56:05.929545 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.929501    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-rules/0.log"
Jul 10 02:56:05.950220 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:05.950148    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65bfd8447f-g69pl_521a4d1c-c832-4ecb-a552-4ea144f712d0/kube-rbac-proxy-metrics/0.log"
Jul 10 02:56:07.603305 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603264    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"]
Jul 10 02:56:07.603794 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603776    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603795    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603825    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603833    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603846    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603851    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603859    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.603865 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603865    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.604078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603934    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.604078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603942    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.604078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603948    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7351c9c5-c0f2-4fee-9c7f-47424d98fc50" containerName="istio-proxy"
Jul 10 02:56:07.604078 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.603957    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="44883dda-6118-4430-b867-1ffd4e596b5b" containerName="istio-proxy"
Jul 10 02:56:07.607840 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.607818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.610405 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.610386    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hdm6f\"/\"kube-root-ca.crt\""
Jul 10 02:56:07.611592 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.611570    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hdm6f\"/\"default-dockercfg-zhgkd\""
Jul 10 02:56:07.611648 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.611570    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hdm6f\"/\"openshift-service-ca.crt\""
Jul 10 02:56:07.615759 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.615685    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"]
Jul 10 02:56:07.723785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.723719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-lib-modules\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.723785 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.723791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-podres\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.723999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.723812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-proc\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.723999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.723896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-sys\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.723999 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.723967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzc76\" (UniqueName: \"kubernetes.io/projected/2d496e4a-3270-49df-bd29-cf7b30f04666-kube-api-access-tzc76\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825112 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-sys\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzc76\" (UniqueName: \"kubernetes.io/projected/2d496e4a-3270-49df-bd29-cf7b30f04666-kube-api-access-tzc76\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-lib-modules\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-podres\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-sys\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825303 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-proc\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-lib-modules\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-podres\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.825507 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.825310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d496e4a-3270-49df-bd29-cf7b30f04666-proc\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.832950 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.832927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzc76\" (UniqueName: \"kubernetes.io/projected/2d496e4a-3270-49df-bd29-cf7b30f04666-kube-api-access-tzc76\") pod \"perf-node-gather-daemonset-hzhsg\" (UID: \"2d496e4a-3270-49df-bd29-cf7b30f04666\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:07.918887 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:07.918815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:08.052387 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:08.052362    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"]
Jul 10 02:56:08.054665 ip-10-0-140-223 kubenswrapper[2579]: W0710 02:56:08.054628    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d496e4a_3270_49df_bd29_cf7b30f04666.slice/crio-e5fb8695f8a0dbf6bca8bf89259c46b55d3b2390c2101483fa9d33117e93202b WatchSource:0}: Error finding container e5fb8695f8a0dbf6bca8bf89259c46b55d3b2390c2101483fa9d33117e93202b: Status 404 returned error can't find the container with id e5fb8695f8a0dbf6bca8bf89259c46b55d3b2390c2101483fa9d33117e93202b
Jul 10 02:56:08.364623 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:08.364594    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fbwrd_1d1bc1f5-da85-4c86-bc33-6f6cef5ee6fb/download-server/0.log"
Jul 10 02:56:09.041299 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:09.041263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg" event={"ID":"2d496e4a-3270-49df-bd29-cf7b30f04666","Type":"ContainerStarted","Data":"543ae61e721dc03f012738a7e70fca6ea7509f2fd274a5dce3ee612f89ac1736"}
Jul 10 02:56:09.041810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:09.041315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg" event={"ID":"2d496e4a-3270-49df-bd29-cf7b30f04666","Type":"ContainerStarted","Data":"e5fb8695f8a0dbf6bca8bf89259c46b55d3b2390c2101483fa9d33117e93202b"}
Jul 10 02:56:09.041810 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:09.041407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:09.058823 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:09.058777    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg" podStartSLOduration=2.058763084 podStartE2EDuration="2.058763084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:56:09.056692171 +0000 UTC m=+2494.353274037" watchObservedRunningTime="2026-07-10 02:56:09.058763084 +0000 UTC m=+2494.355344859"
Jul 10 02:56:09.981996 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:09.981971    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zsqwg_c6fa3dcb-01a3-47a5-8df7-65c08b4cced5/dns/0.log"
Jul 10 02:56:10.001649 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:10.001628    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zsqwg_c6fa3dcb-01a3-47a5-8df7-65c08b4cced5/kube-rbac-proxy/0.log"
Jul 10 02:56:10.050717 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:10.050686    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bbdlw_071228ff-2484-40ba-84cd-49677845886d/dns-node-resolver/0.log"
Jul 10 02:56:10.643836 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:10.643809    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ds69v_0d059ee8-9a60-4470-9110-1bf7ea3e0359/node-ca/0.log"
Jul 10 02:56:11.589181 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:11.589147    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjn9c_e3cbfcff-26cb-4ac3-8f89-245707fab97c/istio-proxy/0.log"
Jul 10 02:56:11.698880 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:11.698853    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmf6d_c93ea990-d141-4a68-90cc-f07a1e17228c/discovery/0.log"
Jul 10 02:56:11.776358 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:11.776327    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/0.log"
Jul 10 02:56:11.807991 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:11.807957    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/1.log"
Jul 10 02:56:12.603148 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:12.603120    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rbbw6_2c747519-bdd7-4d10-b8ea-8ddccddee4dc/serve-healthcheck-canary/0.log"
Jul 10 02:56:13.136025 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:13.135996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcx59_4adcdc5b-595f-4a8b-8315-fc2fb442f8ef/kube-rbac-proxy/0.log"
Jul 10 02:56:13.155502 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:13.155480    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcx59_4adcdc5b-595f-4a8b-8315-fc2fb442f8ef/exporter/0.log"
Jul 10 02:56:13.176227 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:13.176201    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kcx59_4adcdc5b-595f-4a8b-8315-fc2fb442f8ef/extractor/0.log"
Jul 10 02:56:15.062395 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:15.062367    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-hzhsg"
Jul 10 02:56:15.604259 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:15.604232    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vprs7_624c704a-7936-4aa0-9c76-e5f9080f2209/server/0.log"
Jul 10 02:56:15.735316 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:15.735284    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-2f4rw_3435c26f-6547-4282-bc9f-f0117d64ee06/manager/0.log"
Jul 10 02:56:15.751968 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:15.751943    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-2f4rw_3435c26f-6547-4282-bc9f-f0117d64ee06/copy-manifests/0.log"
Jul 10 02:56:17.165568 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:17.165537    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-865944fd75-mktk9_cdcdefb7-f388-4521-934e-b3317d09e02f/manager/0.log"
Jul 10 02:56:18.261161 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:18.261130    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjn9c_e3cbfcff-26cb-4ac3-8f89-245707fab97c/istio-proxy/0.log"
Jul 10 02:56:18.369193 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:18.369159    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmf6d_c93ea990-d141-4a68-90cc-f07a1e17228c/discovery/0.log"
Jul 10 02:56:18.422336 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:18.422309    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/0.log"
Jul 10 02:56:18.452008 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:18.451980    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/1.log"
Jul 10 02:56:19.761598 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:19.761561    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678bjn9c_e3cbfcff-26cb-4ac3-8f89-245707fab97c/istio-proxy/0.log"
Jul 10 02:56:19.866356 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:19.866322    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-kmf6d_c93ea990-d141-4a68-90cc-f07a1e17228c/discovery/0.log"
Jul 10 02:56:19.917420 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:19.917394    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/0.log"
Jul 10 02:56:19.949298 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:19.949261    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-gk5qh_a173f714-4d5d-4d17-bdfb-7e69ef636c13/istio-proxy/1.log"
Jul 10 02:56:23.798085 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.798001    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/kube-multus-additional-cni-plugins/0.log"
Jul 10 02:56:23.818460 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.818434    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/egress-router-binary-copy/0.log"
Jul 10 02:56:23.839491 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.839469    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/cni-plugins/0.log"
Jul 10 02:56:23.862039 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.862017    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/bond-cni-plugin/0.log"
Jul 10 02:56:23.882044 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.882021    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/routeoverride-cni/0.log"
Jul 10 02:56:23.903990 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.903967    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/whereabouts-cni-bincopy/0.log"
Jul 10 02:56:23.924664 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:23.924641    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kg4rd_88c33e16-a136-4dae-a73f-73901b742b10/whereabouts-cni/0.log"
Jul 10 02:56:24.136635 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:24.136555    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-drrlv_f548050d-50ab-4a9d-99dd-f8a0a78a5e70/kube-multus/0.log"
Jul 10 02:56:24.156008 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:24.155982    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7mx5l_88c8b447-df78-4fb5-9932-c7154765ee06/network-metrics-daemon/0.log"
Jul 10 02:56:24.175561 ip-10-0-140-223 kubenswrapper[2579]: I0710 02:56:24.175540    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7mx5l_88c8b447-df78-4fb5-9932-c7154765ee06/kube-rbac-proxy/0.log"